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Introduction and objectives 
of the handbook 

In recent years, significant progress has been achieved in delivering health-related interventions that are 
designed to achieve goals relating to improving maternal and child health, and reducing mortality and ill- 
health due to HIV/AIDS, tuberculosis and malaria. It is increasingly apparent, however, that the gains have been 
neither universal nor sufficiently broad-based and sustainable. Progress at the national level has not necessarily 
resulted in gains for most vulnerable population groups; in some instances, progress has stagnated or been 
reversed. There is mounting evidence that health systems that can deliver services equitably and efficiently are 
critical for achieving improved health status. Thus, many global health initiatives now incorporate attention to 
health systems strengthening in the support they provide to countries. 

While this increased attention to the strengthening of health systems is welcome, it would not be sustainable 
in the absence of a sound monitoring strategy that enables decision-makers to accurately track health progress 
and performance, evaluate impact, and ensure accountability at country and global levels. Moreover, the use 
of results-based financing mechanisms by major global donors has created a further demand for timely and 
reliable data. There is also increasing in-country demand for data in the context of annual health sector reviews. 

Information is needed to track how health systems respond to increased inputs and improved processes, and 
the impact they have on improved health indicators. This implies the need to define core indicators of health 
system performance while developing and implementing appropriate sustainable measurement strategies to 
generate the required data. However, on the supply side, there are major gaps in data availability and quality. 
Few developing countries are able to produce data of sufficient quality to permit the regular tracking of progress 

in scaling-up health interventions and 
strengthening health systems. Data gaps 
span the range of “input”, “process”, 
“output”, “outcome” and “impact” 
indicators: e.g. few countries carry out 
regular national health accounts studies; 
data on the availability and distribution 
of health workers are often incomplete, 
inaccurate and out of date; few countries 
have systems that can monitor service 
delivery; and data on population access 
to essential services are limited. 
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What is a health system? 

A health system consists of all the organizations, institutions, resources and people whose primary purpose 
is to improve health. 1,2 This includes efforts to influence determinants of health as well as more direct 
health-improvement activities. The health system delivers preventive, promotive, curative and rehabilitative 
interventions through a combination of public health actions and the pyramid of health care facilities that 
deliver personal health care — by both State and non-State actors. The actions of the health system should be 
responsive and financially fair, while treating people respectably. A health system needs staff, funds, information, 
supplies, transport, communications and overall guidance and direction to function. Strengthening health 
systems thus means addressing key constraints in each of these areas. 

Frameworks for monitoring health systems performance 

The multifaceted nature of health systems and the spread of direct and indirect responsibilities across multiple 
sectors, pose challenges in monitoring performance. In response, over the past several years, the World Health 
Organization (WHO) and its partners have been working to reach a broad-based consensus on key indicators 
and effective methods and measures of health systems capacity, including “inputs”, “processes” and “outputs”, 
and to relate these to indicators of “outcome”. It is widely known that there are many potential advantages of 
a harmonized approach to health systems monitoring and evaluation, including reduced transaction costs, 
increased efficiency, and diminished pressures on countries. However, there are also identified practical issues 
to be addressed before greater harmonization can become a reality. The existence of multiple analytical and 
strategic frameworks for health systems results in considerable potential for duplication, overlap and confusion. 3 
Existing frameworks include the WHO framework for health systems performance assessment ( 1 ); the World 
Bank control knobs framework ( 2 ); and the WHO building blocks framework ( 3 ). Such frameworks have varying 
starting points, resulting in emphases on different outcomes to be tracked. Work is on to develop conceptual 
frameworks for health systems strengthening and to create a taxonomy that would permit clarification of the 
indicators, data sources and collection methods, and the analytics underpinning monitoring and evaluation. 
However, the choice of the strategic framework does not necessarily substantively affect the monitoring and 
evaluation strategy. There are many commonalities in the various strategic frameworks for health systems that 
permit a coherent approach to the choice of indicators and measurement strategies. 

Health systems framework and building blocks 

This handbook does not attempt to cover all components of the health system or deal with the various monitoring 
and evaluation frameworks. Instead, it is structured around the WHO framework that describes health systems 
in terms of six core components or “building blocks”: (i) service delivery, (ii) health workforce, (iii) health 
information systems, (iv) access to essential medicines, (v) financing, and (vi) leadership/governance (see 
Figure 1). 


1 World Health Organization, http://www.who.int/healthsystems/about/en/ accessed June 2010. 

2 The terms “health system” and “health sector” are often used interchangeably with the latter interpreted as restricted to the actions 
of the government. This handbook focuses on aspects of the health system that are under the responsibility of ministries of health, 
including the provision of personal health services by both State and non-State actors. 

3 For a recent overview see Shakerishvili G. Building on health systems frameworks for developing a common approach to health systems 
strengthening. Prepared for the World Bank, Global Fund to Fight AIDS, Tuberculosis and Malaria, and GAVI Alliance, Technical 
Workshop on Health Systems Strengthening, Washington, DC, June 25-27,2009. 
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Figure 1. The WHO Health Systems Framework 
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Source: (3) 

The six building blocks contribute to the strengthening of health systems in different ways. Some cross-cutting 
components, such as leadership/governance and health information systems, provide the basis for the overall 
policy and regulation of all the other health system blocks. Key input components to the health system include 
specifically, financing and the health workforce. A third group, namely medical products and technologies and 
service delivery, reflects the immediate outputs of the health system, i.e. the availability and distribution of care. 

Inevitably, any type of division of a complex construct such as the health system is fraught with problems. This 
is also true for the framework, which focuses on health sector actions and underplays the importance of actions 
in other sectors. It does not take into account actions that influence peoples’ behaviours, both in promoting and 
protecting health and the use of health-care services. The framework does not address the underlying social 
and economic determinants of health, such as gender inequities or education, and also does not deal with the 
substantial and dynamic links and interactions that exist across each component. 

On the other hand, focusing on these separate components helps put boundaries around this complex construct 
and permits the identification of indicators and measurement strategies for monitoring progress. 

Towards a common monitoring and evaluation framework 

Interest in a common monitoring and evaluation framework was stimulated as a result of the International 
Health Partnership and related initiatives (IHP+). 4 Launched in September 2007, the IHP+ aims to better 
harmonize donor funding commitments, and improve the way in which international agencies, donors and 
developing countries work together to develop and implement national health plans. The IHP+ has developed a 
common monitoring and evaluation framework to enable targeted monitoring and evaluation of health system 
strengthening efforts (Figure 2). This framework is country-focused and supportive of country needs while also 
providing a basis for global monitoring. 


4 For more information, visit http://www.internationalhealthpartnership.net/en/home, accessed May 21,2010. 
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The monitoring and evaluation framework shows how health inputs and processes (e.g. health workforce and 
infrastructure) are reflected in outputs (e.g. interventions and available services) that in turn are reflected in 
outcomes (e.g. coverage) and impact (morbidity and mortality). The added value of the framework is that it 
brings together indicators and data sources across the results chain in its entirety, i.e. from “inputs/processes”, 
“outputs”, and “outcomes”, to “impact”. It is designed to address monitoring and evaluation needs for different 
users and multiple purposes, including: 

• monitoring of programme inputs, processes and results, required for the management of health system 
investments; 

• health systems performance assessment, as the key for country decision-making processes; and 

• evaluating the results of health reform investments and identifying which approaches work best. 


Figure 2. Monitoring and evaluation of health systems strengthening 
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Scope of this handbook 

The handbook discusses each building block separately according to a common format, with the medical 
products, vaccines and technologies building block focusing specifically on access to essential medicines. 
For each component or building block, the handbook identifies a parsimonious set of indicators and related 
measurement strategies. This handbook does not address the measurement strategies and indicators across 
the entire results chain of the common monitoring and evaluation framework. Rather, it focuses on systems 
“inputs,” “processes” and “outputs” as they relate to each of the six building blocks of health systems. 

The indicators were initially identified by a small working group consisting of agency representatives and 
technical experts and then subsequently shared more broadly with country experts, and supported with case- 
studies and reviews of country experiences. The handbook describes indicators, measurement approaches and 
strategies that: 

• permit the establishment of country health system statistical profiles; 

• permit the monitoring of health systems and guides country and partner investments; 

• highlight gaps in terms of data availability and quality, pointing to needed investments in measurement 
strategies; and 

• contribute to a global consensus on how to monitor and benchmark health systems strengthening. 

In view of the dynamic interlinkages and cross-cutting nature of the different components, the indicators may 
also have multiple relevance. For example, availability of data on human resources for health may also be an 
appropriate measure of core information system functions. 

The selection of indicators was guided by the need to detect change and show progress in health systems 
strengthening. Indicators relate to both the level and distribution of inputs and outputs. While the focus is 
on low- and middle-income countries, experiences from high-income countries are also used to guide the 
development of measurement systems. 

Health systems are complex and their performance and impact are difficult to capture using only quantitative 
indicators. A complete report of health system performance must ensure that quantitative indicators are 
complemented by qualitative information; however, this topic is not addressed in this handbook. 

Structure and content of the handbook 

The handbook is divided into six sections, each of which covers one health system component or building block 
and is set out along the following lines: 

• introduction to the component and related indicators; 

• description of possible sources of information and available measurement strategies; 

• proposed “core indicators”, supplemented, where necessary, by additional indicators that may be used 
depending on the country health system attributes and needs. 

A summarized list of core indicators for monitoring each health system component or building block is 
presented as an Annex at the end of this section. 
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In the sections on leadership/governance, service delivery and information systems, composite indices, derived 
from a selection of system indicators, are presented. These cover both core and additional indicators, reflecting 
the scope and the complexity involved in trying to measure multiple subsystem components. Issues related 
to improving data availability and quality, with consideration of the investments that may be needed, are also 
presented in the service delivery and health workforce sections. 

Core principles guiding the use of this handbook 

When working with countries to measure and compare health systems functioning, it is important to strike a 
good balance between avoiding blueprints that do not allow for country contexts and specificities while also 
encouraging a degree of standardization that enables comparisons within and between countries as well as over 
time. Standardized indicators allow comparisons between countries and can help mutual learning, including 
the identification of bottlenecks and the sharing of lessons learnt. In general, however, measurement should be 
attuned to a country’s health strategy objectives. Each section has proposed core indicators that all countries are 
encouraged to collect, plus a wider set of indicators that users can choose or modify as needed. It is anticipated 
that the core indicators will enable the production of country “dashboards” that contain the instruments 
by which health systems trends can be regularly monitored and compared. Countries should integrate new 
indicators with existing indicators of their health sector and statistical strategies and plans. Health systems 
monitoring should also be seen in the context of the indicators’ impact on access to priority health services and 
their contribution to reaching the Millennium Development Goals (MDGs). 

A number of the proposed indicators require disaggregation by sub-populations or units, e.g. by sex, age 
or location. Often such sub-analyses are country specific. Continued research and knowledge generation is 
needed to inform and generate evidence that would help in understanding the actual meaning of trends in an 
indicator’s value. 

Strategies and investments for improving data availability and quality 

Responsibilities for effective data generation go beyond that of ministries of health and involve other bodies, 
such as departments and agencies that handle health-related data, including national statistics offices, ministries 
of education, among others. There is a need for a strong coordinating body that brings together the various 
stakeholders and helps ensure the development of a comprehensive and integrated plan for health information and 
statistical system development. Such a plan should provide the basis for enhanced alignment and harmonization of 
technical and financial support from the development partners. The Health Metrics Network (HMN) framework 
has set out standards for information system components and data management (5). 

Having poor or unreliable data creates long-term costs and unforeseen effects that are high compared with the 
costs and benefits of having good data. Practical and generally affordable strategies exist for generating timely 
and reliable data on health systems, but appropriate investment is needed to develop the capacity to collect, 
manage, analyse, disseminate and use the obtained information. Further work on the costs of generating data 
required for monitoring health systems strengthening is needed. 
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Table: List of recommended core indicators 


Building blocks and indicators 

Data collection methods / Data sources 

1. Health Service Delivery 

• Number and distribution of health facilities per 10 000 population 

District and national databases of health facilities. Special 

• Number and distribution of inpatient beds per 10 000 population 

efforts — notably facility censuses — are often reguired 
to obtain the number of private facilities, especially if no 
registration system is enforced. 

• Number of outpatient department visits per 10 000 population per year 

Routine health facility reporting system 

Population-based surveys 

• General service readiness score for health facilities 

• Proportion of health facilities offering specific services 

• Number and distribution of health facilities offering specific services per 10 000 population 

• Specific-services readiness score for health facilities 

Health facility assessments 

2. Health Workforce 

• Number of health workers per 10000 population 

Routine administrative records, periodically validated and 

• Distribution of health workers by occupation/specialization, region, place of work and sex 

adjusted against data from national population census or 
facility-based assessments. 

• Annual number of graduates of health professions educational institutions per 100000 
population, by level and field of education 

Routine administrative records from individual training 
institutions. In some cases, data may be validated 
against registries of professional regulatory bodies where 
certification or licensure is reguired for practice. 

3. Health Information 

• Health information system performance index 

Review of national health information systems 


Continues... 
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Continued 


4. Essential Medicines 

• Average availability of 14 selected essential medicines in public and private health facilities 

• Median consumer price ratio of 14 selected essential medicines in public and private health 
facilities 

National (or sub-national when necessary) surveys of 
medicine price and availability conducted using a standard 
methodology developed by WHO and Health Action 
International. 

5. Health Financing 

• Total expenditure on health 

• General government expenditure on health as a proportion of general government 

National Health Accounts (NHA) 

expenditure (GGHE/GGE) 

• The ratio of household out-of-pocket payments for health to total expenditure on health 

Household expenditure and utilization surveys. 

6. Leadership and Governance 

Policy index 

Review of national health policies in respective domains 
(such as essential medicines and pharmaceutical, 

TB, malaria, HIV/AIDS, maternal health, child health/ 
immunization). 
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1. Health service delivery 




Health service delivery 

1.1 Introduction 

Strengthening service delivery is crucial to the achievement of the health-related Millennium Development 
Goals (MDGs), which include the delivery of interventions to reduce child mortality, maternal mortality and 
the burden of HIV/AIDS, tuberculosis and malaria. Service provision or delivery is an immediate output of the 
inputs into the health system, such as the health workforce, procurement and supplies, and financing. Increased 
inputs should lead to improved service delivery and enhanced access to services. Ensuring availability of health 
services that meet a minimum quality standard and securing access to them are key functions of a health system. 

To monitor progress in strengthening health service delivery, it is necessary to determine the dimensions along 
which progress would be measured. Box 1.1 sets out eight key characteristics of good service delivery in a 
health system. These ideal characteristics describe the nature of the health services that would exist in a strong 
health system based on primary health care, as set out in the 2008 World Health Report (1). 

The process of building evidence for the strengthening of health service delivery must therefore proceed 
alongside efforts to restructure service delivery in accordance with the values reflected in Box 1.1. Health 
sector leaders and policy-makers who are tasked with assessing their health systems should participate in the 
process to deliberate on ways to assess these key characteristics in their countries. Researchers should continue 
to experiment with methods and measures that would allow progress to be assessed over time, along these 
important dimensions. 

For some of the dimensions of service delivery, such as quality of care, widely accepted methods and indicators 
for assessment are available, although research to refine these continues. For other characteristics in the list, 
such as person-centredness, research and dialogue on what and how to measure it is in the early stages. 

Some concepts that have frequently been used to measure health services remain extremely relevant and are 
part of the key characteristics. For example, terms such as access, availability, utilization and coverage have often 
been used interchangeably to reveal whether people are receiving the services they need (2, 3). Access is a broad 
term with varied dimensions: the comprehensive measurement of access requires a systematic assessment of the 
physical, economic, and socio-psychological aspects of peoples ability to make use of health services. Availability 
is an aspect of comprehensiveness and refers to the physical presence or delivery of services that meet a minimum 
standard. Utilization is often defined as the quantity of health care services used. Coverage of interventions is 
defined as the proportion of people who receive a specific intervention or service among those who need it. 
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Box 1.1: Key characteristics of good service delivery 


Good service delivery is a vital element of any health system. Service delivery is a fundamental 
input to population health status, along with other factors, including social determinants of 
health. The precise organization and content of health services will differ from one country to 
another, but in any well-functioning health system, the network of service delivery should have 
the following key characteristics. 

1. Comprehensiveness: A comprehensive range of health services is provided, appropriate 
to the needs of the target population, including preventative, curative, palliative and 
rehabilitative services and health promotion activities. 

2. Accessibility: Services are directly and permanently accessible with no undue barriers of cost, 
language, culture, or geography. Health services are close to the people, with a routine point 
of entry to the service network at primary care level (not at the specialist or hospital level). 
Services maybe provided in the home, the community, the workplace, or health facilities as 
appropriate. 

3. Coverage: Service delivery is designed so that all people in a defined target population are 
covered, i.e. the sick and the healthy, all income groups and all social groups. 

4. Continuity: Service delivery is organized to provide an individual with continuity of care 
across the network of services, health conditions, levels of care, and over the life-cycle. 

5. Quality: Health services are of high quality, i.e. they are effective, safe, centred on the 
patient’s needs and given in a timely fashion. 

6. Person-centredness: Services are organized around the person, not the disease or the 
financing. Users perceive health services to be responsive and acceptable to them. There is 
participation from the target population in service delivery design and assessment. People are 
partners in their own health care. 

7. Coordination: Local area health service networks are actively coordinated, across types 
of provider, types of care, levels of service delivery, and for both routine and emergency 
preparedness. The patients primary care provider facilitates the route through the needed 
services, and works in collaboration with other levels and types of provider. Coordination 
also takes place with other sectors (e.g. social services) and partners (e.g. community 
organizations). 

8. Accountability and efficiency: Health services are well managed so as to achieve the core 
elements described above with a minimum wastage of resources. Managers are allocated 
the necessary authority to achieve planned objectives and held accountable for overall 
performance and results. Assessment includes appropriate mechanisms for the participation 
of the target population and civil society. 


This section of the handbook focuses particularly on the physical availability of services, which may serve as a 
starting point for determining methods to improve service delivery. It presents the measurement strategies and 
indicators for monitoring as well as the “inputs”, “processes” and “outputs” to the health system as they relate to 
the service delivery building block (see Figure 2 in the Introduction section). 

Service delivery monitoring has immediate relevance for the management of health services, which distinguishes 
this area from other health systems building blocks. Shortage of medicines, uneven distribution of health 
services, and the poor availability of equipment or guidelines must all be taken into account as part of basic 
service management. 
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1.2 Sources of information on health service delivery 

There are multiple sources of data on health service delivery. These include routine facility reporting systems, health 
facility assessments (both facility censuses and surveys), and other special studies. No single method provides all 
the information required to assess service delivery, and multiple methods are needed to understand it completely. 
The strengths and limitations of the different methods are summarized in Table 1.1 and discussed below. 

Routine health facility reporting system 

A routine facility reporting system, often referred to as a Health Management Information System (HMIS), 
is generally used to monitor service delivery. Service data are generated at the facility level and include key 
outputs from routine reporting on the services and care offered and the treatments administered. Reporting 
may include supervisory or clinic-reported data on medicine stock-outs in a defined reference period (e.g. 
during the last month), functioning of outreach services and availability of health workers. Because the data 
are routinely collected (often monthly or quarterly), it provides information on a continuous basis for time and 
seasonal trend analyses. 

The problems associated with developing service coverage estimates from facility data relate to completeness and 
accuracy of recording and reporting as well as biases arising from differences in use of services by different populations. 
In general, routine facility reporting systems give only limited information on the status of service delivery. In many 
settings, the HMIS often covers only public sector facilities (which may include not-for-profit facilities). 


Table 1.1 Summary of main methods of collecting data on service delivery 


Data collection 
method 

Description 

Strengths 

Limitations 

Routine health 
facility reporting 
system 

Regular facility data reported to 
regional and national levels by 
service providers 

Mandated practice at the facility level 
with standard reporting formats and 
cycles 

Limited data on service provision; often 
incomplete, covers public sector only, and 
with time lags in reporting; biases due to 
variation in population use of services 

Health facility 
census 

Periodic census of all public and 
private health-care facilities 
within a country 

Provides information useful to 
planners at all levels, such as basic 
characteristics (ownership, facility 
type, coordinates), availability and 
functionality of basic infrastructure, 
staffing, service provision and general 
status 

Time-consuming and can become costly, if 
not well integrated; difficult to identify all 
health-care facilities, particularly in urban 
centres where smaller private practices 
may be more common; access to all 
facilities may be problematic 

Health facility 
survey 

Periodic survey of a 
representative sample of public 
and private health-care facilities 
within a country 

More detailed information than in 
facility census with verification of 
information in many cases; guality 
of care 

Time-consuming and costly; information 
most useful at national level; reguires a 
complete facility listing for sampling to 
be done correctly; long intervals between 
surveys 


Hospital records are the basis for statistics on performance related to inpatient activities, including the numbers 
of beds, admissions, discharges, deaths and the duration of stay. Outpatient records are the basis for utilization 
data. As with other routine facility reporting, problems arise from incomplete and late reporting as well as from 
biases resulting from differences in population use of services. 
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Health facility assessments 

Health facility assessments provide externally generated information either through interviews and/or 
observation for data collection. Health facility assessments can be implemented as a census (i.e. assessment of 
all facilities in a district or country) or by using a sample survey approach (i.e. a sample of facilities are selected 
and assessment). 

Facility census 

A facility census includes visits to all public and private health facilities in a defined area (can be national in 
scope or sub-national, covering one or more provinces, regions or districts). It is designed to form the basis for 
a national and sub-national monitoring system of service delivery. The key output is a national database, and 
where possible, district databases of health facilities. The database should be updated on a regular basis, e.g. 
every 3-4 years. Once a reliable database system (that can be used at the district level) is in place, the census can 
be carried out by district teams as part of their regular supervision, with a quality control component provided 
by regional teams. 

The World Health Organization (WHO) service availability and readiness assessment methodology provides a 
standard health facility assessment questionnaire to assess, map and monitor service availability and readiness 
(4). It is designed to support a health facility census with a focus on the core functional capacities and availability 
of services. The instrument can be further adapted at the country level to respond to specific country contexts. 
If resources are limited and do not allow for visiting all health facilities in a country (or sub-nationally in a 
district, region, or province), a census can be implemented in sentinel districts with additional districts added 
each year, to achieve a full census over a longer time period. 

The key topic areas and core functional capacities of a facility census of service availability and readiness include: 

• Identification, location and managing authority of health facility (public and private) 

• Facility infrastructure and amenities, such as availability of water supply, telecommunications and 
electricity 

• Basic medical equipment, such as weighing scales, thermometer and stethoscope 

• Availability of health workforce (e.g. cadre of human resources, staff training and guidelines) 

• Drugs and commodities — availability of general medicines 

• Diagnostic facilities — availability of laboratory tests (e.g. HIV, malaria, tuberculosis (TB), others) 

• Standard precautions on prevention of infections — availability of general injection and sterilization, 
disposal and hygiene practices 

• Specialized services, such as family planning, maternal and newborn care, child health, HIV/AIDS, 
tuberculosis, malaria and chronic diseases. 

Facility censuses also serve as an independent source for numbers of health workers, which maybe compared 
with those from other sources and analysed in conjunction with them. Additional particulars, such as the 
presence of workers on the day of the visit, can also be gathered. Comparisons between districts and regions 
provide valuable evidence about the distribution of services within a country. Information on minimum 
standards can be used for key services to provide feedback to programme planners. 

The identification of all facilities, however, is a major challenge. Small private facilities are more likely to be 
missed, and special efforts have to be made to include them, especially in urban areas. Completeness is likely to 
improve with subsequent rounds of censuses. Other sources, such as household surveys in which respondents 
are asked which facilities they utilize, maybe used to identify more centres. Obtaining access to private facilities 
for the brief interview can pose another challenge. 
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A facility census can only check on the basic elements of service quality. In general, no data are collected on patient 
satisfaction or knowledge and practices of health workers, as this would be very time-consuming and costly Thus, 
quality ascertainment could only be achieved through facility surveys and further in-depth assessments. 

Facility surveys 

A general facility survey usually focuses on a wide range of key health services and collects information on 
facility infrastructure, equipment and supplies, support systems, management systems and providers’ adherence 
to standards. 

Facility surveys may also measure the quality of specific services and whether all required elements are present to 
provide routine care; for example, immunization and diarrhoea treatment in the survey of child health services. 
The core questionnaire reflects generally accepted standards for health-care services, including United Nations 
Childrens Fund (UNICEF) immunization guidelines and standards set by the Safe Motherhood initiative, with 
local adaptations as necessary. 

The United States Agency for International Development (USAID) and Macro International Inc. have developed 
a comprehensive facility survey instrument called Service Provision Assessment. 1 The survey is conducted in a 
nationally representative sample of health facilities (often exceeding 400 facilities, stratified by type) to provide 
information on the characteristics of health services, including their quality, infrastructure, utilization and 
availability. The assessment covers all types of health service sites, from hospitals to health posts, including public 
and private institutions. Data collection includes facility resources audit, provider interviews, client-provider 
observations and client exit interviews. Another example of a comprehensive facility assessment is the “balanced 
scorecard” in Afghanistan used to monitor the scale-up of health services (5, 6), as described in Box 1.2. 



Box 1.2 Facility survey with a balanced scorecard, Afghanistan 


A “balanced scorecard” approach was developed in Afghanistan to monitor the scale-up of health 
services. The assessment relies on a facility survey, including health worker interviews, client- 
provider observations and exit interviews to assess the perception of quality and satisfaction 
with services. 

Six domains and 29 indicators were used and monitored through annual surveys during 
2004-2006. The domains included patients and community (e.g. patient satisfaction), staff 
(e.g. salary payments), capacity for service provision (e.g. equipment functionality, medicine 
availability, training intensity, and infrastructure), service provision (e.g. proper sharps disposal 
and outpatient visits per month), financial systems (e.g. user fee guidelines and exemptions for 
poor patients) and overall vision (e.g. outpatient department visit concentration index). 


The objective of a facility survey is not to provide information on the strengths and weaknesses for specific 
facilities, but to identify the strengths and weaknesses in health systems. The findings can be used to measure 
changes in the systems put in place to support quality services and adherence to standards. The facility survey 
presents information not only on the availability of services, but also on measures of quality. 


1 http://www.measuredhs.com/aboutsurveys/spa/start.cfm 
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One of the disadvantages of the facility survey, however, is the cost of obtaining extensive information whose 
relevance is only at the national level. The extensive data collection efforts in each facility provide a wealth of 
information on hundreds of indicators, but a much smaller number of indicators matter for policy-making. 
Moreover, the utility of the information on the quality of care is hampered by the bias inherent in exit surveys, 
which are by their nature limited to recent users of care and do not constitute a population-based sample. 

1.3 A service delivery monitoring system 

Given the strengths and weaknesses of each data source, it is clear that no single source can provide sufficient 
information for monitoring service delivery. Thus, a service delivery monitoring system would need to rely on 
multiple sources of data to be brought together for analysis and decision-making. Data from routine health 
facility reporting systems need to be supplemented with data from health facility assessments. The topics 
included in these assessments will vary over time and the questionnaire should use a modular approach selected 
on the basis of current priorities and needs. In addition, data generated through facility assessments should 
be complemented or cross-checked with data from other sources, such as the databases of health workers, 
infrastructures, equipment and procurement, that are often available in various departments of the ministries 
of health. This can serve as a complementary or benchmarking material for data on service delivery generated 
through the routine HMIS. 

Information, regardless of the source, should preferably be collected and made available at the district level. 
Ideally, the foundation of a system of monitoring health resources lies at the district level, as it provides 
information required for decision-making. Therefore, establishing a district-based system is the primary goal 
with support at the national or regional/provincial levels. In the context of decentralization, provinces are often 
given the responsibility for monitoring and evaluation, but little investment is made to assist them in carrying 
out this role. By investing at the provincial level, an independent monitoring system that provides essential data 
for the district level and allows comparison between districts can be set up. 

1.4 Core indicators 

Countries have often defined their own set of performance measures in the area of service delivery. The challenge 
is to devise a set of sensitive and specific indicators that can easily be collected at all facilities at relatively little 
cost, and with the possibility of becoming part of regular facility reporting systems. 

This section lists a small set of service delivery indicators focused on low-income and lower middle-income 
countries (Table 1.2). The purpose of this set is to monitor the strength of the health system over time. Although 
in some a single tracer condition may suffice for monitoring purposes, it is generally important to bring together a 
range of indicators to summarize the overall situation; thus summary measures or indices are useful. The indicators 
listed below can be grouped into those that reflect general service availability, general service readiness, and service- 
specific availability and readiness. 
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Table 1.2 Summary of proposed core indicators to monitor service delivery 


Core Indicators 

Data collection method 

General service availability 

la Number and distribution of health facilities per 10 000 population 

National database of health facilities 
(often reguiring facility censuses) 

1 b Number and distribution of inpatient beds per 10 000 population 

1 c Number of outpatient department visits per 10 000 population per year 

Routine health facility reporting system 

Population-based surveys 

General service readiness 

2a General service readiness score for health facilities 

Health facility assessments 

Service-specific availability 

3a Proportion of health facilities offering specific services 

Health facility assessments 

3b Number and distribution of health facilities offering specific services 
per 10 000 population 


Service-specific readiness 

4a Specific-services readiness score for health facilities 

Health facility assessments 


1.4.1 General service availability 

General service availability refers to the physical presence of delivery of services that meet a minimum standard. 
Availability comprises health infrastructure (facilities and beds per 10 000 population), the health workforce 
per 10 000 population and aspects of service utilization (inpatient/outpatient visits per 10000 population). 

Recommended indicator la: Number and distribution of health facilities per 10 000 
population 

Definition 

The number of health facilities available relative to the total population for the same geographical area. 

• Numerator: the number of health facilities, i.e. all public and private health facilities, defined as a static 
facility (a designated building) in which general health services are offered. It does not include mobile 
service delivery points and non-formal services, such as traditional healers. 

• Denominator: the total population for the same geographical area. 

Data collection methodology 

District and national databases provide the number of public facilities, often by type (such as hospital, health 
centre, health post, dispensary). Special efforts, notably facility censuses, are often required to obtain the 
number of private facilities, especially if no registration system is enforced. A facility sample survey will not 
provide the data needed to compute service availability. 

Comparability issues 

The size of health facilities may vary considerably and affect comparisons. When smaller geographical units, 
such as districts are analysed, the population does not necessarily use the facilities in the designated area. 
Comparisons of densities between districts have to be made cautiously. 
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Periodicity 

Annual updating of the number of facilities, and validation every 3-5 years through a complete census. 

Complementary dimensions 

Distribution implies urban-rural differences and could also include differences between regions or provinces, 
or sometimes between districts. Since the population size of districts tends to be small, comparisons of densities 
between districts have to be made cautiously. 

Additional information can be presented based on the managing authority of health facilities: public, private 
not-for-profit (including faith based), private for profit, and other (such as parastatals). 

Recommended indicator 7b; Number and distribution of inpatient beds per 70 000 
population 

Definition 

The number of inpatient beds available relative to the total population for the same geographical area. 

• Numerator: the number of inpatient beds. This includes total hospital beds (for long-term and acute care), 
maternity beds and paediatric beds, but not delivery beds. Public and private sectors are included. 

• Denominator: the total population for the same geographical area. 

Data collection methodology 

District and national databases provide the number of beds. Special efforts, notably facility censuses, are often 
required to obtain the number of beds in private facilities, especially if no registration system is enforced. 

Comparability issues 

When smaller geographical units, such as districts are analysed, the population does not necessarily use the 
facilities in the designated area. Comparisons of densities between districts have to be made cautiously. 

Periodicity 

Regular updating of the number of beds in facilities, and validation every 3-5 years through a complete census. 

Complementary dimensions 

Distribution implies urban-rural differences and could also include differences between regions or provinces, 
or sometimes between districts. 

Additional information about beds can be presented based on the managing authority of health facilities: public, 
private not-for-profit (including faith based), private for profit, and other (such as parastatals). 

Data on maternity beds can also be used to calculate the density of maternal beds per 1000 pregnant women 
per year. 

Indicators of service availability cannot, of course, accurately reflect access to services. True indicators of 
access need to measure the proportion of the population living within a specified travel time and/or distance 
from a health facility. Access is sometimes measured through household surveys which rely on respondent 
judgment or on basic spatial analysis of catchment areas around specified facilities. However, the latter is 
subject to weaknesses. For example, designing catchment areas around health facilities (e.g. a 5 km or 10 km 
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buffer) without considering the capacity of each health facility and logistic constraints for patients, provides 
only a rough estimate of physical access. Moreover, such calculations require data on the global positioning 
system coordinates of all service delivery points and population data for small geographical areas. Only a 
limited number of countries are conducting such analyses on a regular basis. In addition, problems arise when 
patients use facilities that he outside the immediate catchment area, which could be due to number of reasons 
including the logistics of travel, sociocultural preferences and perceptions of quality. Urban areas present a 
particular challenge because although physical proximity may pose less of a problem, issues of affordability and 
acceptability become more important obstacles to access. 

Recommended core indicator 1c: Number of outpatient department visits per 10 000 
population per year 

The patient volumes at inpatient and outpatient facilities are not a coverage indicator because the population 
in need is not well defined. Low rates, however, are indicative of poor availability and quality of services. For 
example, several countries have demonstrated that outpatient department rates go up when constraints to using 
health services are removed, such as by bringing services closer to the people or reducing user fees. In contrast, 
once rates exceed an uncertain threshold the number of visits is no longer an indicator of the strength of the 
health services. 

Definition 

The number of outpatient visits to health facilities relative to the total population of the same geographical area 

• Numerator: the number of visits to health facilities for ambulant care, not including immunization (can be 
divided into children under five years of age and aged five years and over). 

• Denominator: the total population for the same geographical area 

Data collection methodology 

• Routine health facility reporting system 

• Population-based surveys 

Comparability issues 

The accuracy and completeness of reporting need to be consistent over time and between populations to allow 
assessment of trends and comparisons. 

Periodicity 

Annual statistics. 

Complementary dimensions 

Disaggregation by district or province/region can be presented. 

Potential additional indicators of inpatient care and utilization include admission rates (number of new 
admissions per 10 000 population per year) and number of caesarean sections per 100 deliveries. Both indicators 
tend to vary considerably, however, with country practices and changes in admission or intervention policies. 
Very low rates tend to indicate that services are not available, but otherwise the statistics are difficult to interpret. 


© Health service delivery 


Two related indicators are: 

1. Average length of stay: an indicator of quality and efficiency of health services. 

2. Bed occupancy rate: an indicator of efficiency of services. 

1.4.2 General service readiness 

General service readiness refers to the general capacity of health facilities to provide health services. Readiness 
is defined as the cumulative availability of components required to provide services. It comprises tracer items 
for the following major domains: infrastructure/amenities, basic supplies/equipment including small surgery, 
standard precautions, laboratory tests, medicines and commodities. 

Recommended core indicator 2a: General service readiness score for health facilities 

Estimation of general service readiness is derived from data on availability and functioning of tracer items in the 
facility on the day of assessment. These items are grouped in five domains (see Table l.A.l in the Annex to this 
section). 

Definition 

Cumulative availability of components required in health facilities to provide general services, expressed as 
percentage. The overall score is the unweighted average of domain scores. Each domain score represents the 
average number of items present and functioning in the health facilities, expressed as a percentage of the total 
number of items in that domain. 

Data collection methodology 

Health facility assessments using a standardized questionnaire to assess the availability and functioning of 
the tracer items in each domain of general service readiness (e.g. WHO core tool for Service Availability and 
Readiness Assessment). 

Comparability issues 

Definitions and data collection should be standardized. 

Periodicity 

The basic state of facilities should be an important input into annual health sector reviews and monitored 
annually at the national level through sample surveys. Districts may use the index as a management tool. 

Complementary dimensions 

Scores for the different domains of general service readiness should be presented separately. All scores can be presented 
by district and by ownership of facilities. 

Hospitals provide a wide range of services and an expanded version of the questionnaire should be administered 
during facility assessment, or regular reporting. 
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1.4.3 Service-specific availability 

Service-specific availability refers to whether or not a specific service is offered. Availability is captured by the 
proportion of services offering a specific service and the density of the facilities offering the service per 10 000 
population. 

Specific services may include family planning, antenatal care, safe delivery, child health, HIV/ AIDS, tuberculosis, 
malaria, chronic conditions and small surgery. 

Recommended core indicator 3a: Proportion of health facilities offering specific 
services 

Recommended core indicator 3b: Number and distribution of health facilities offering 
specific services per 70 000 population 

Definition 

Proportion of health facilities ready to provide key services: The number of facilities that offer specific services 
relative to the total number of facilities. 

Number and distribution of health facilities ready to provide key services per 10 000 population: The number of 
facilities that offer specific services relative to the total population in the same geographical area. 

• Numerator: the number of facilities in which a specific service is offered. 

• Denominator: the total number of facilities (to obtain the proportion), or, the total population for the same 
geographical area (to compute the density) 

Data collection methodology 

Health facility assessments using a standardized questionnaire to assess the availability and functioning of the 
components required to meet the key service capacity standards. 

Comparability issues 

Definitions and data collection should be standardized. When using indices, the results for the specific 
components should be specified. 

Periodicity 

The state of facilities should be monitored annually at the sub-national level as a management tool. National 
statistics should be updated every 2-3 years through regular reporting by districts; and sample surveys and a 
census once every 3-5 years to validate all information. 

Complementary dimensions 

Distribution implies urban-rural differences and could also include differences between regions or provinces, 
or sometimes between districts. 

1.4.4 Service-specific readiness 

Service-specific readiness refers to the capacity of health facilities to provide a specific service, measured 
through the presence of tracer items that include trained staff, guidelines, equipment/supplies, diagnostic 
capacity, medicines and commodities. The main challenge is to develop a concise set of items for each domain 
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so that all programmes can be monitored through a single data collection mechanism. Recommended tracer 
items to monitor readiness to deliver specific services are presented in Table 1 .A.2 in the Annex to this section. 

Recommended core indicator 4: Service-specific readiness score for health facilities 

Definition 

Cumulative availability of components required in health facilities to deliver specific services, expressed as 
percentage. The overall score for a specific service is the unweighted average of number of items present and 
functioning, expressed as a percentage of the total number of items in that service. 

Data collection methodology 

Health facility assessments using a standardized questionnaire to assess the availability and functioning of the 
tracer items in each service provided. 

Comparability issues 

Definitions and data collection should be standardized. 

Periodicity 

The basic state of facilities should be an important input into annual health sector reviews and monitored 
annually at the national level through sample surveys. Districts may use the index as a management tool. 

Complementary dimensions 

Scores for the different domains of general service readiness should be presented separately. All scores can be 
presented by district and by ownership of facilities. 

Hospitals provide a wide range of services and an expanded version of the questionnaire should be administered 
during facility assessment, or regular reporting. 

1.4.5 Service quality 

Assessing quality of care can be difficult because it can cover both the complex processes of evaluating, 
diagnosing and treating a patient as well as the outcomes of that treatment for the patient. In most definitions, 
quality of care is seen to be multidimensional: care is said to be of high quality if it is effective, safe, centred on 
the patients needs and given in a timely fashion. 

A wide range of quality indicators is presented in Table 1. A.3 in the Annex to this section, but measurement and 
monitoring are major challenges. The Organisation for Economic Co-operation and Development (OECD) has 
identified 13 indicators that could be used in several of its countries to assess the quality of health-care delivery 
(7). Most of these indicators are related to health outcomes or coverage of interventions, e.g. six indicators 
on cancer screening and survival ratios (breast, cervical and colorectal cancer) and three on vaccination 
(coverage, impact). Waiting time to intervention for femur fractures and inpatient 30-day case fatality rates 
for acute myocardial infarction and stroke are indicators that are more directly related to service organization 
and quality. The USAID-sponsored Quality Improvement Project has examined a range of quality data in 
developing countries relevant to compliance with specific care standards as well as outcome indicators such as 
case-fatality rates for specific diseases, e.g. diarrhoeal disease or measles. WHO has worked the world over as 
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part of its IMCI programme to assess the level of compliance with the IMCI algorithm, specifically in areas of 
assessment and treatment protocols. Many countries are examining the use of admission rates for conditions, 
such as asthma and diabetes, that are readily manageable in the primary care setting. 

In many parts of the world, data on quality of care are very difficult to obtain. However, death registers and 
reviews of mortality for case-fatality rates are feasible sources of information in many settings. Admission rates 
for certain conditions amenable to care in the primary care setting are also available. Other sources of data 
include: 

Quality assurance practices 

• Supervisory checklist for health services: presence of equipment and completeness of HMIS accounts, and 
other process indicators 

• Supervisory checklist for health service provision: contents in the client assessments, treatments or 
consultations 

• Facility-wide review of mortality: structured system to review the records of each death 

• Audits of medical records or registers: checking if protocols are followed. 

Supportive management practices 

• Facility supervisory visit in last six months 

• Report of provider receiving routine pre-service or in-service training, personal supervision. 

Funding mechanism 

• Routine user fee for adult curative care 

• Any external source for reimbursement (insurance, employers, charity). 

Logistics 

• Adequate system for monitoring temperature of vaccine stock: functioning thermometer in refrigerator, 
up-to-date temperature chart, temperature 0-8°C at the time of survey 

• Vaccine stock: no expired items, items stored by expiration date, up-to-date inventory available 

• The DELIVER Project’s Logistic Information Assessment Tool has developed a core set of indicators with 
data available for many countries (8). 

Good storage and stock monitoring systems for medicines (contraceptives) 

• Good storage conditions: dry location, off the ground, protected from water, sun, pests and rodents 

• Adequate stock monitoring: no expired items, items stored by expiration date, up-to-date inventory available. 

Selected tools 

Profiles of health facility assessment methods. Report of the International Health Facility Assessment Network 
(IHFAN). Arlington, VA, MEASURE Evaluation/USAID, 2008 (http://www.cpc.unc.edu/measure/publications/ 
pdf/tr-06-36.pdf, accessed 11 June 2010). 


^ Health service delivery 


This document profiles four instruments used for health facility assessment, and specifies their management 
utility The instruments included are Service Provision Assessment, Facility Audit of Service Quality, Health 
Facility Census, and Service Availability Mapping. 

Health systems assessment approach: a how-to manual. Bethesda, MD, Health Systems 20/20 (http://www. 
healthsystems2020.org/content/resource/detail/528/, accessed 11 June 2010). 

This manual is designed to provide a rapid yet comprehensive assessment of a country’s health system. The 
approach covers key health systems functions and is organized around seven technical modules, which 
guide data collection and assessment, including that of service delivery. 

World Health Organization. Service availability and readiness assessment methodology, forthcoming (http:// 
www.who.int/healthinfo/systems/serviceavailabilitymapping/en/, accessed 30 August 2010) 

The service availability and readiness assessment methodology is a standard health facility assessment 
methodology developed by WHO to assist countries to assess, map and monitor services availability and 
readiness at health facility levels (including hospitals, health centres, pharmacies and laboratories). The 
tool is implemented in countries to help create a baseline database of all public and private health facilities 
and services across the country or in sentinel districts, to form the basis for a national and sub-national 
routine monitoring system of service delivery. 

Further reading 

Kelley E, Hurst J. Health Care Quality Indicators Project conceptual framework paper. Paris, Organisation for 
Economic Co-operation and Development, 2006 (OECD Health Care Working Papers, No. 23, DELSA/HEA/ 
WD/HWP(2006)3; http://www.oecd.Org/dataoecd/l/36/36262363.pdf, accessed 11 June 2010). 

USAID. Logistics indicators assessment tool (LIAT). Arlington, VA, USAID DELIVER Project, 2008 (http:// 
deliver.jsi.com/dhome/topics/monitoring/monitoringpubs/meresources/metools, accessed 11 June 2010). 

USAID. Assessment tool for laboratory services (ATLAS) 2006. Arlington, VA, DELIVER for the United 
States Agency for International Development, 2006 (http://deliver.jsi.com/dhome/topics/monitoring/ 
monitoringpubs/meresources/metools, accessed 11 June 2010). 
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Annex 

Table 1 .A.1 Tracer items for assessment of general service readiness 


Domain 

Tracer items 

1. Basic amenities 

• Power (a grid or functional generator with fuel) 

• Improved water source within 500 meters of facility 

• Room with auditory and visual privacy for patient consultations 

• Access to adequate sanitation facilities 

• Communication equipment (phone or SW radio) 

• Access to computer with email/internet 

• Emergency transport 

2. Basic equipment 

• Adult scale 

• Child/infant scale 

• Thermometer 

• Stethoscope 

• Sphygmomanometer and BP cuff 

• Refrigerator 

• Light source 

3. Standard precautions for prevention 
of infections 

• Sterilization equipment 

• Storage and disposal of sharps 

• Storage and safe disposal of infectious wastes 

• Disinfectant 

• Sharps box/container 

• Single use - standard disposable or auto-disable syringes 

• Soap or hand disinfectant 

• Latex gloves 

• Masks 

• Guidelines for standard precautions 

4. Laboratory 

• Hemoglobin 

• Whole blood glucose by glucometer 

• HIV rapid test 

• Rapid syphilis test 

• Malaria rapid test or smear 

• TB microscopy (by AFT light microscopy) 

• General microscopy (e.g. wet mounts) 

• Urine pregnancy rapid test 

• Urine dipstick 

5. Medicines and commodities 

• Standard 14 essential medicines (see table 4.1) 

Total 

Overall general service readiness index 
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Table 1 .A.2 Tracer items to monitor readiness to deliver specific services 


Services 

Tracer items 

1. Family planning services 

Staff and training 

• Guidelines on family planning 

• Stafftrained in FP 

Eguipment 

• Blood pressure machine 

• Stethoscope 

Medicines and commodities 

• Combined oral contraceptive pills 

• Injectable contraceptives 

• Condoms (male) 

2. Antenatal care services 


Staff and training 

• Guidelines on ANC 

• Stafftrained in ANC 

Eguipment 

• Blood pressure machine 

• Stethoscope 

Diagnostics 

• Hemoglobin 

• Urine protein 

Medicines and commodities 

• Iron tablets 

• Folic acid tablets 

• Tetanus toxoid 


3. Basic emergency obstetric and newborn care 

Staff and training • Guidelinesfor Integrated management of pregnancy and childbirth (IMPAC) 



• Stafftrained in IMPAC 

Eguipment 

• Emergency Transport 

• Examination light 

• Suction apparatus 

• Manual vacuum extractor 

• Vacuum aspirator or D&C kit 

• Newborn bag and mask 

Medicines and commodities 

• Partograph 

• Gloves 

• Antibiotic eye ointment for newborn 

• Injectable uterotonic 

• Injectable antibiotic 

• Magnesium sulphate 

• Intravenous solution with infusion set 

Health service delivery 
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Services 


Tracer items 


4. Comprehensive Emergency Obstetric Care (CEmO) 

Staff and training 

• Guidelines for CEmOC 

• Stafftrained in CEmOC 

• Surgeon and anesthetist on staff 

Equipment 

• Anesthesia equipment 

• External heat source 

Diagnostics 

• Blood typing capacity 

Medicines and commodities 

• No shortage of blood in last three months; blood obtained ONLY from national or regional blood bank OR 

Blood obtained from other sources but screened for HIV and other transfusion transmissible infections 

5. Child health services: routine child immunization 

Staff and training 

• Guidelines for EPI 

• Stafftrained in EPI 

Equipment 

• Cold box with ice packs 

• Refrigerator 

Medicines and commodities 

• Syringes and needles 

• Sharps box 

• Measles vaccine 

• DPT-HB vaccine 

• Polio vaccine 

• BCG vaccine 

6. Child health services: curative care and preventive services including growth monitoring 

Staff and training 

• Guidelines for IMCI 

• Stafftrained in IMCI 

Equipment 

• Child/infant scale 

• Thermometer 

• Growth charts 

Diagnostics 

• Hemoglobin (Hb) 

• Test parasite in stool 

• Malaria blood test 

Medicines and commodities 

• Oral Rehydration Solution packet 

• Amoxicillin 

• Co-trimoxazole 

• Paracetamol 

• Vitamin A 

• Me-/albendazole 

• Zinc 


Continues... 
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Services 

Tracer items 

7. HIV counseling and testing 


Staff and training 

• Guidelines on HIV testing 

• Guidelines on HIV & AIDS counseling 

• Stafftrained in HIV testing 

• Stafftrained in HIV & AIDS counseling 

Equipment 

• Visual and auditory privacy 

Diagnostics 

• HIV diagnostic test 

• Trained HIV/AIDS diagnostic provider 

Medicines and commodities 

• Condoms 

8. HIV/AIDS care and support services 

Staff and training 

• Guidelines for clinical management of HIV& AIDS 

• Guidelines for palliative care 

• Stafftrained in clinical management of HIV & AIDS 

Diagnostics 

• System for diagnosis of TB among HIV + clients 

Medicines and commodities 

• Intravenous solution with infusion set 

• IV treatment fungal infections 

• Cotrimoxazole 

• First-line TB treatment medications 

• Palliative care pain management 

• Condoms 

9. HIV/AIDS antiretroviral prescription and client management 

Staff and training 

• Guidelines for antiretroviral therapy 

• Stafftrained in ART prescription and management 

Diagnostics 

• Complete blood count (CBC) 

• CD4, Viral Load, or Total Lymphocyte Count (TLC) 

• Blood urea levels 

• Liver function test (LFT) 

Medicines and commodities 

• All first-line antiretrovirals 

10. Preventing mother-to-child transmission (PMTCT) services 

Staff and training 

• Guidelines for PMTCT 

• Stafftrained in PMTCT 

• Guidelines for infant and young child feeding counseling 

• Stafftrained in infant and young child feeding 

Equipment 

• Visual and auditory privacy 

Diagnostics 

• HIV diagnostic test for adults 

• Dried blood spot (DBS) filter paper for diagnosing newborn HIV+ 

Medicines and commodities 

• Zidovudine (AZT) 

• Nevirapine (NVP) 
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Services 

Tracer items 

11. Tuberculosis (TB) services 


Staff and training 

• Guidelines for diagnosis and treatment ofTB 

• Guidelines for management of HIV&TB co-infection 

• Guidelines related to MDR-TB treatment (or identification of need for referral) 

• Stafftrained in TB diagnosis and treatment 

• Stafftrained in management of HIV &TB co-infection 

• Stafftrained in client MDR-TB treatment or identification of need for referral 

Diagnostics 

• TB smear microscopy 

• Trained malaria diagnostic provider 

• HIV Test 

Medicines and commodities 

• First-lineTB medications 


12. Malaria services-if relevant* (*only for high prevalence regions) 


Staff and training 

• Guidelines for diagnosis and treatment of malaria 

• Staff trained in malaria diagnosis and treatment 

• Guidelines for IPT* 

• Service provider trained in IPT* 

Diagnostics 

• Malaria diagnostic capacity 

• Trained malaria diagnostic provider 

Medicines and commodities 

• At least two first-line antimalarials in stock 

• IPT drug* 

• ITN* 


13. Chronic Diseases adult treatment and preventive services 

Staff and training • Guidelines for diabetes diagnosis and treatment 



• Guidelines for mental health treatment 

• Guidelines for treatment of chronic cardio-vascular conditions 

• Stafftrained in management of chronic illnesses 

Eguipment 

• Stethoscope 

• Sphygmomanometer and BP cuff 

• Adult weight scale 

Diagnostics 

• Capacity to measure blood glucose 

Medicines and commodities 

• Enalapril. Tracer drugs: Atenolol (tenormin) 50 mg capsule/tablet; Captopril (capoten) 25 mg capsule/tablet 

• Beclometasone inhaler. Tracer druges: Salbutamol 0.1 mg/dose inhaler 

• Metformin tablet. Tracer drugs: Glibendamide (5 mg capsule/table 

• Insulin 

• Amitriptyline 25 mg capsule/tablet 


Continues... 
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Services 

Tracer items 

14. Minor surgery services 

Staff and training 

• Needle holder 


• Scalpel handle with blade 


• Retractor 


• Surgical scissors 


• Nasogastric tubes 10-16 FG 


• Tourniguet 

Medicines and commodities 

• Skin disinfectant 


• Sutures (both absorbable and non-absorbable) 


• Ketamine 


Table 1 .A.3 Sample indicators for consideration in assessing health-care quality 

Dimension of care 

Indicators 

Data source 

Effectiveness 

Case-fatality rates for specific diseases 

Hospital admission rate for asthma 

Percentage of sick child visits during which health worker counseled mother on 
nutrition 

Percentage of women aged 40 years and over who reported a mammogram 
within the past two years 

Percentage of women who received prenatal care in the first trimester 

Record review 

Record review 

Observation, exit interviews 

Survey 

Record review or survey 

Safety 

Percentage of providers who know hand hygiene guidelines 

Birth trauma rate in neonate per 1000 live births 

Percentage of adults whose provider asks about other prescribed medication 

Interviews with health workers 

Record review 

Observation, exit interviews 

Patient-centredness 

Percentage of adults with recent health visit who stated their provider always 
listened to what they had to say 

Percentage of adults with recent health visit who stated their provider explained 
things clearly 

Percentage of adults with recent health visit who stated their provider showed 
respect to them 

Exit interviews, household survey 

Exit interviews, household survey 

Exit interviews, household survey 

Timeliness 

Percentage of persons who state they have a usual source of care 

Percentage of emergency department visits where patients left without being 
seen 

For heart attack patients, median time to thrombolytic therapy or percutaneous 
transluminal coronary angioplasty (PTCA) 

Survey 

Record review 

Laboratory records 
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2. Health workforce 




Health workforce 

2.1 Introduction 

The ability of a country to meet its health goals depends largely on the knowledge, skills, motivation and 
deployment of the people responsible for organizing and delivering health services. Numerous studies show 
evidence of a direct and positive link between the numbers of health workers and population health outcomes 
(1, 2). Many countries, however, lack the human resources needed to deliver essential health interventions for 
a number of reasons, including limited production capacity, migration of health workers within and across 
countries, poor mix of skills and demographic imbalances. The formulation of national policies and plans 
in pursuit of human resources for health development objectives requires sound information and evidence. 
Against this backdrop of an increasing demand for information, building knowledge and databases on the health 
workforce requires coordination across sectors. WHO is working with countries and partners to strengthen the 
global evidence base on the health workforce — including gaining consensus on a core set of indicators and a 
minimum data set for monitoring the stock, distribution and production of health workers. 

The health workforce can be defined as “all people engaged in actions whose primary intent is to enhance 
health” (3). These human resources include clinical staff, such as physicians, nurses, pharmacists and dentists, 
as well as management and support staff, i.e. those who do not deliver services directly but are essential to the 
performance of health systems, such as managers, ambulance drivers and accountants (Box 2.1). Presently, 
comprehensive and robust methodologies are not available for assessing the adequacy of the health workforce to 
respond to the health-care needs of a given population. However, a shortage of health workers can be perceived 
from the inadequate numbers and skills mix of people being trained or maldistribution of their deployment, as 
well as losses caused by death, retirement, career change or out-migration. It has been estimated that countries 
with fewer than 23 physicians, nurses and midwives per 10 000 population generally fail to achieve adequate 
coverage rates for selected primary health-care interventions, as prioritized by the MDGs (3). 



Box 2.1 Boundaries of the health workforce 

Various permutations and combinations of what constitutes the health workforce may exist 
according to the country’s situation and the means of monitoring. Human resources for health 
include individuals working in the private and public sectors, those working full-time or part- 
time, those working at one job or holding jobs at two or more locations, and those who are paid or 
provide services on a voluntary basis. They include workers in different domains of health systems, 
such as curative, preventive and rehabilitative care services as well as health education, promotion 
and research. They may also include people with the education and training to deliver health 
services but who are not engaged in the national health labour market (e.g. if they are unemployed 
or have migrated or withdrawn from the labour force for personal reasons). 
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The need for comprehensive, reliable and timely information on human resources for health, including numbers, 
demographics, skills, services being provided and factors influencing recruitment and retention, has been 
widely identified at the international, regional and national levels among both resource-poor and wealthier 
countries. This need has become even more urgent in view of the international effort to scale-up education and 
training of health workers in 57 countries, mostly in sub-Saharan Africa, which have been identified as having 
a critical shortage of highly skilled health professionals (3). 

A health information system with a strong human resources component can help build the evidence base to plan 
for the availability of required health workers of desired quality in the right place, at the right time. Planning 
requires knowledge of the numbers and characteristics of health workers who are active in the health sector, 
of those being trained and added to the human resources pool, and of those leaving the active workforce and 
their reasons for leaving (4, 5). A comprehensive Human Resources Information System (HRIS) can also guide 
decision-making to ensure the cultural appropriateness of the health system, such as the appropriate sex and 
ethnic mix of health workers, especially to encourage utilization of services among underserved or marginalized 
communities. For example, in some contexts, access to female providers is an important determinant of women’s 
health service utilization patterns (6). A strategy for ensuring the male-female balance of the health workforce 
should include promoting the collection and use of sex-disaggregated data in all human resource assessments. 

A timely, reliable and relevant HRIS is essential to support the formulation, monitoring and evaluation of health 
workforce plans, strategies and policies at the sub-national, national and international levels. Unfortunately, for 
most countries, there remains a significant lag between the demand for data and the availability and usefulness 
of the information required to support decision-making. 

2.2 Sources of information on the health workforce 

Effective monitoring and evaluation of human resources for health in countries requires the development of 
an agreed core set of indicators and their means of measurement to inform decision-making among national 
authorities and other stakeholders. Diverse sources that can potentially produce relevant information exist 
even in low-income countries, such as population-based sources, health facility assessments and routine 
administrative records (4, 5, 7-11). Each of these sources has its strengths and limitations for health workforce 
analysis (Table 2.1). In many countries, comprehensive data on human resources are not available in any one 
repository. This means that any attempt to determine the size and core characteristics of the health workforce 
requires some level of analysis and synthesis of available information from multiple sources. The use of 
information from a variety of sources should, in principle, increase the options for measuring and validating 
core health workforce statistics. 

Population censuses and surveys 

Many meaningful results pertinent to workforce analysis can be produced through tabulation of population- 
based data. All countries collect at least some data on their population, mainly in terms of periodical 
demographic censuses and household sample surveys that produce statistical information about the people, 
their homes, their socioeconomic conditions and other characteristics. Most censuses and labour force surveys 
ask for the occupation and place of work of the respondent (and other adult household members) along with 
other demographic characteristics, including age, sex and education levels. 
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Nationally representative population censuses and labour force surveys with properly designed questions on 
occupation, place of work and field of training allow the identification of people with education and training 
in health, those in health-related occupations and those employed in health services industries. Enumeration 
of health workers from census data is a count of the number of people with a health-related occupation and/ 
or working in the health services industry. A similar method is used for counting health workers from labour 
force survey data, with the additional application of a sampling weight to calibrate for national representation. 

Health facility assessments 

Health facility assessments can be conducted using different sampling approaches (establishment census or 
sample survey) and methodologies (self-administered postal, fax or Internet-based questionnaire, or telephone 
or face-to-face interview). Depending on the nature of the data collection procedures and instruments, in depth 
information can be obtained on a range of health workforce variables, such as in-service training, support 
supervision and current staffing levels in relation to planned staffing norms. A census-based sampling approach 
may be better suited for collecting data on the numbers and distribution of facility-based health workers, while 
an approach that uses sampling and subsequent extrapolation may be better for collecting data on health worker 
motivation and productivity. In addition, the nature of facility-based assessments helps in the collection of data 
for numerous other indicators pertinent to health systems performance assessment, such as infrastructure, 
availability of supplies and costs. 

Administrative records 

In many countries, the computerization of administrative records — including public expenditure, staffing 
and payroll, work permits, health insurance and social security records — is greatly facilitating the possibilities 
for analysis. The administrative records of health training institutions and professional licensing bodies are 
potentially valuable sources for tracking the health workforce as many skilled health-care providers require 
formal training, registration and licensure to practice their professions. These sources offer the advantage 
of producing continuously updated statistics. In addition, depending on the characteristics of the registries, 
notably where individuals are assigned a unique identifier, it may be possible to track workers’ labour force 
entry, career progression and exit. 

Data comparability and synthesis across multiple information sources 

Precisely defining and classifying the health workforce remains an important challenge for comparing 
information across sources, countries, and over time. Health workers play different roles and often have 
different national history, culture and codes of practice. For example, nursing and midwifery personnel 
may be characterized by different educational requirements, legislation and practice regulations, skills and 
scope of practice between countries (and even within a given country). Countries with critical shortages and 
maldistribution of highly skilled medical and nursing professionals, have large numbers of non-physician 
clinicians (often called clinical officers or surgical technicians) and/or community health workers (12, 13), for 
whom the levels of training vary widely. Comparability of health worker data can be enhanced through the 
setting and use of common definitions and classifications for monitoring the labour market (4,5). The collection, 
processing and dissemination of information should follow internationally standardized classifications for 
social and economic statistics (or their national equivalent), including the International Standard Classification 
of Occupations (ISCO), International Standard Industrial Classification of all Economic Activities (ISIC) and 
International Standard Classification of Education (ISCED) (14-16). Details of the uses of these frameworks 
for statistical delineation of the health workforce are presented in the Annex to this section. 

In particular, the ISCO enables occupations to be arranged into a hierarchical system according to the skill level 
and skill specialization required to carry out the tasks and job duties. In the latest 2008 ISCO revision (known 
as ISCO-08), most health occupations are expected to fall within two sub-major groups: group 22 “health 
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professionals” (generally well-trained workers in jobs that normally require a university degree for competent 
performance) and group 32 “health associate professionals” (generally requiring knowledge and skills acquired 
through advanced formal education but not equivalent to a university degree). This distinction was designed to 
reflect differences in tasks and duties that may be a consequence of differences in work organization as well as 
education and training. It must be recognized, however, that in some countries the possibility of distinguishing 
between the two typologies of nurses and midwives remains limited; inadequacies in the reporting system or 
incomparability of the education systems and measures of technical capacity may mean that some nursing and 
midwifery jobs do not fit easily into these two categories. 

The main statistical advantages of ISCO are in the setting of clearly defined occupational groups within and 
across countries and in monitoring the migration of workers between countries. Overall, it is expected that 
possibilities for health workforce analysis will be strengthened in the upcoming 2010 round of population 
censuses, which will be able to exploit the new ISCO-08 revision (17). Among the significant improvements in 
ISCO-08 (compared with the previous version adopted in 1988) is the creation of new unit codes for identifying 
more types of health service providers, including paramedical practitioners and community health workers, as 
well as certain categories of health management and support workers (notably health service managers, health 
information technicians and medical secretaries). The March 2008 revision to the WHO Global atlas of the 
health workforce reflects the improved classification (18) (Box 2.2). Ideally, information on these categories of 
workers should be available for all countries where the occupations are practiced. 



Box 2.2 Counting health workers: occupational categories in the Global atlas of the health 
workforce 


The classification of health workers used for the WHO Global atlas of the health workforce (18) 
is based on criteria for vocational education and training, regulation of health professions, and 
activities and tasks of jobs, i.e. a framework for categorizing key workforce variables according to 
shared characteristics. The WHO framework largely draws on the latest revision to the International 
Standard Classification of Occupations (ISCO) and other standard classifications for social and 
economic statistics. Data on nine occupational categories are captured in the main data set: 

1. Physicians 

2. Nursing and midwifery personnel 

3. Dentistry personnel 

4. Pharmaceutical personnel 

5. Laboratory health workers 

6. Environmental and public health workers 

7. Community and traditional health workers 

8. Other health service providers 

9. Health management and support workers 

National-level data are collated from four main sources: population censuses, labour force surveys, 
health facility assessments and administrative reporting systems. Where available, disaggregated 
data are presented on up to 18 occupational categories as well as on the distribution of the health 
workforce by age, sex and geographical location (urban/rural). 
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Tools that aim at providing guidelines on how to develop national classifications and their mapping 
to international standards are available. One such manual, developed by the European Centre for the 
Development of Vocational Training and Eurostat, based on an analysis of the descriptions of the content of 
training programmes, is intended to serve as a guide for countries where comprehensive national classifications 
for vocational education and training are not developed (19). The most up-to-date information from the 
International Labour Organization and latest advice for countries on how to develop, maintain and revise a 
national occupation classification and its mapping to the international standard can be found on the ISCO 
web site (14). 

Given the diversity of information sources for health workforce monitoring, it is especially important that 
the dissemination of statistics concerning human resources for health include metadata descriptors for each 
data point, including details on its nature and coverage. This would be crucial for efforts to synthesize and 
triangulate figures across multiple sources, in particular to distinguish whether the data include: health workers 
in the private sector, workers who are unpaid or unregulated but performing health-care tasks, potential 
double-counts of workers holding two or more jobs at different locations, or trained health service providers 
not currently working at health facilities or other service delivery points. 

2.3 Core Indicators 

Recommended core indicator 7; Number of health workers per 70 000 population 

The health worker density — the number of health workers per 10 000 population, by cadre — is the health 
workforce indicator that is most commonly reported internationally and represents a critical starting point for 
understanding the health system resources situation in a country. When measured systematically, this indicator 
provides information on the stock of health workers relative to the population. It can be used to monitor whether, 
for example, the size of the current workforce meets a given threshold that should allow the most basic levels of 
health-care coverage to be achieved across the country. The advantages are that it is simple to calculate, maybe 
used for comparative analyses across countries and over time, and is easy to understand among a wide range of 
audiences, facilitating its usefulness for advocacy purposes. However, it does not necessarily take into account 
all health system objectives, particularly with regard to accessibility, equity, quality and efficiency. 

Definition 

The number of health workers available in a country relative to the total population. 

• Numerator: the absolute number of health workers at a given time in a given country or region (that is, all 
persons eligible to participate in the national health labour market by virtue of their skills, age, ability and 
physical presence in the country). 

• Denominator: the total population for the same geographical area. 

Data collection methodology 

Ideally assessed through routine administrative records on numbers of active health workers compiled, 
updated and submitted regularly (e.g. quarterly) by district health officers, payroll registrars, individual health 
facilities (both public and private) and/or health professional regulatory bodies, and collated into a centralized 
HRIS or database maintained by the ministry of health or other mandated agency. Information on the stock of 
health workers and on the total population should be periodically validated and adjusted against data from a 
population census or other nationally representative source. 
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Comparability issues 

Data on health occupations should ideally be classified according to the latest ISCO revision (or its national 
equivalent). 

Periodicity 

Monthly, quarterly or annually for routine administrative records. A validation exercise should be conducted 
every 3-5 years against a national population-based or facility-based assessment. 

Complementary dimensions 

The most complete and comparable data currently available on the health workforce globally pertain to 
physicians, nurses and midwives. However, the health workforce includes a wide range of other categories of 
service providers (e.g. dentists, pharmacists, community health workers) as well as management and support 
workers (health service managers, health economists, health information technicians and others). Information 
on all of these categories of human resources for health should ideally be captured. 

Recommended core indicator 2: Distribution of health workers - by occupation / 
specialization, region, place of work and sex 

There is increasing interest globally in equity in health and the pathways by which inequities arise and are 
perpetuated or exacerbated. Imbalance (or maldistribution) in the supply, deployment and composition of 
human resources for health, leading to inequities in the effective provision of health services, is an issue of 
social and political concern in many countries. Drawing on an analytical framework for understanding health 
workforce imbalance (20), at least four typologies for monitoring the distribution of health workers should 
be considered: (i) imbalances in occupation/specialty, (ii) geographical representation, (iii) institutions and 
services, and (iv) demographics. As the impact of these different types of imbalances on the health system 
varies, there is a need to monitor and assess each of these dimensions of workforce distribution. In practical 
terms, this implies that the collection, processing and dissemination of health workforce data should enable 
disaggregation by occupation (and within a given occupation, for example by medical specialization), by 
geographical typology (e.g. urban or rural, within or outside the capital city, by province/state or district), 
by place of work (e.g. hospital or primary health-care facility, public or private), by main work activities (e.g. 
preventive/curative/rehabilitative health-care provision versus other functions such as teaching or research), 
and by sex. 

Definition 

The distribution of health workers according to selected characteristics — notably, by occupation, geographical 
region, place of work and sex. 

• Numerator: the number of health workers with a given characteristic (e.g. working in a privately operated 
health facility). 

• Denominator: the total number of health workers. 

Data collection methodology 

The means of measuring the distribution of the health workforce is a simple disaggregation of the stock of 
health workers (see above indicator of density) according to the selected characteristics. 
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Comparability issues 

Data on occupation and place of work should ideally be classified by or mapped to the ISCO and ISIC, 
respectively 

Periodicity 

Monthly, quarterly or annually for routine administrative records. A validation exercise should be conducted 
every 3-5 years against a national population-based or facility-based assessment. 

Complementary dimensions 

Because counts of workers in the private sector are likely to be less accurate when drawing on administrative 
sources than counts of those in the public sector, and because private for-profit providers are often less 
accessible to low-income populations, it is recommended that national and international reports include 
statistics disaggregated by employment sector (public, private for-profit and private not-for-profit). Additional 
information on health workers’ demographic characteristics may also be important for policy and planning, 
e.g. the age distribution can lend insights into the numbers of workers approaching retirement age. 

Recommended core indicator 3: Annual number of graduates of health professions 
educational institutions per 100 000 population - by level and field of education 

Another commonly reported indicator for monitoring health workforce metrics is the annual output (or 
number of graduates) of health professions educational institutions relative to the population (or to the 
current active health workforce). This is actually not one measure but the aggregate of multiple pieces of 
information, depending on the number of cadres in the health system. The number and type of newly trained 
health workers is relevant everywhere: in countries that need increased production among all cadres, in 
countries that need more workers in rural and underserved areas, and in countries receiving large numbers 
of foreign-trained workers that are aiming towards national self-sufficiency of health workforce regeneration. 

Definition 

Number of graduates from health profession educational institutions (including schools of medicine, dentistry, 
pharmacy, nursing, midwifery and other health services) during the last academic year, divided by the total 
population. 

• Numerator: the absolute number of graduates of health professions educational institutions in the past 
academic year (by level and field of education). 

• Denominator: total population. 

Data collection methodology 

Ideally assessed through routine administrative records from individual training institutions (both public and 
private) submitted regularly (e.g. annually) and collated into a centralized HRIS or database maintained by 
the ministry of health or other mandated agency. In some cases, data may be validated against registries of 
professional regulatory bodies where certification or licensure is required for practice. 

Comparability issues 

Data on health worker education and training should ideally be classified by or mapped to the ISCED. 
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Periodicity 

Annually. 

Complementary dimensions 

Data on the output of health professions educational institutions can be used to assess health workforce renewal 
or the ratio of entry to the health workforce (i.e, the number of graduates relative to the total active health 
workforce). When combined with data on the numbers of foreign-trained health workers in the country, this 
information can be used to assess the level of national self-sufficiency in human resources for health. Data from 
school records can also be used to obtain information on student applications, enrolments and attrition, as well 
as institutional capacity and curriculum content, for the production of quality health workers (21). 

2.4 Additional considerations for monitoring national workforce plans and actions 

Strengthening the performance of health systems depends on more than just increasing the numbers of health 
workers; actions for assessing and strengthening their recruitment, distribution, retention and productivity 
are also important. Actions may include: adopting new approaches to pre-service and in-service training; 
strengthening workforce management; establishing or improving incentives for addressing distribution and 
retention challenges; or task-shifting (delegating tasks, where appropriate, to less specialized health workers). 
Such strategic plans should normally include targets for monitoring health workforce metrics in both the short - 
and the long-term and adaptation to any major health sector reforms (for example, decentralization). At the 
same time, the plans should be harmonized with broader strategies for social and economic development (e.g. 
the national poverty reduction strategy paper). They should also focus on the human resources development 
needs of priority health programmes and aim to integrate these into a primary health-care framework, based 
on epidemiological evidence. 

Table 2.2 presents a series of indicators for monitoring human resources dynamics and their potential means of 
verification. Not all of the indicators necessarily require a numerical answer; e.g., the existence of a documented 
human resources management and development plan could be a relevant indicator for providing information 
on a particular strategic direction (22). The list suggested here is neither exhaustive nor absolute, but an attempt 
to build a framework for monitoring and evaluation of health workforce strategies and actions at the country 
level. Disaggregation of relevant indicators allows for monitoring progress in actions to improve equity in 
access and coverage of essential health interventions, especially among underserved communities or other 
nationally prioritized population groups. A number of tools and resources exist to assist countries in setting 
their health personnel needs and targets (23). Approaches should focus not only on health service providers, 
but also the health management and support staff needed to keep systems and services running. 
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Table 2.2 Selected indicators for monitoring country actions for strengthening the health workforce 
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Effective management of performance of health workers. Related Number of senior staff at primary health-care Routine administrative records of training programmes Optional indicator: 

activities include training programmes for updating skills for effective facilities who received in-service management and/or interviews with key informants (e.g. Rate of retention of health service 

human resources management and development training (with nationally approved curriculum) programme managers) providers at primary health-care 

in the past 12 months facilities in the past 12 months 
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It will be imperative to review the present selection of proposed indicators at the national and sub-national 
levels, particularly in the process of establishing appropriate country-specific baselines and targets. It is 
important to keep in mind the need, where possible, to routinely compile, analyse and act on data collected 
through existing administrative processes. This routine data collection can then be supplemented and validated 
through periodic or ad hoc surveys and other standard statistical sources. 

Sharing information is important so that improved human resources strategies can be compared and used 
by others. Intercountry knowledge sharing as part of the HRIS strengthening process provides models that 
help to avoid repeating mistakes and standardizes information and evidence across regions and countries. In 
particular, health workforce observatories are a valuable mechanism that can be used for widely disseminating 
information and evidence for effective practices at the national, regional and global levels (Box 2.3). 



Box 2.3 A mechanism for sharing experiences, information and evidence to support policy 
decision-making: health workforce observatories 


Initiatives for supporting the development, implementation, monitoring and evaluation of human 
resources for health actions and strategies should ensure not only the collection and processing 
of appropriate data, but also dissemination and utilization for policy and managerial decisions. 
Health workforce observatories are being increasingly promoted to improve the translation of 
information and evidence into policy-making and practice by offering a cooperative mechanism 
for countries and partners to produce and share information and knowledge. Although the 
functions of and triggering force for the emergence of health workforce observatories differ across 
countries and regions, depending on specific contexts and needs, all have the common objective 
of bridging the gap between evidence and policies. 

Some examples of health workforce observatories at the national and regional levels include: 

• Africa Health Workforce Observatory (http://www.afro.who.int/hrh-observatory) 

• Eastern Mediterranean Region Observatory on Human Resources for Health (http://www. 
emro.who.int/hrh%2Dobs) 

• Latin America and Caribbean Observatory of Human Resources in Health (http://www. 
observatoriorh.org) 

• Observatorio Andino de Recursos Humanos en Salud (http://bvsde.per.paho.org/oarhs2/ 
index.php) 

• Observatorio de Recursos Humanos en Saude do Brasil (http://www.ObservaRH.org.br) 

• Ghana Health Workforce Observatory (http://www.ghanahrhobservatory.org) 

• Sudan National Human Resources for Health Observatory (http://www.hrhobservatory.sd/) 


To sum up, practical and affordable strategies exist for generating timely and reliable statistics on the health 
workforce and for developing the capacity to collect, manage, analyse and disseminate them (Box 2.4). The cost 
of not improving workforce statistics is much higher than that of investing in these strategies: poorly informed 
decisions and unmonitored interventions can have long-term social and economic effects, which is critical 
because impacts of interventions and effects of adjustments can sometimes take several years to be observable 
(up to eight years in the case of producing physicians). 
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Box 2.4 Financial resource needs for a timely and comprehensive human resources 
information system 

In practical terms, the cost of collecting and processing nationally representative data on the health 
workforce will be marginal for exercises that already include questions on occupation, education 
and place of work (e.g. population census or labour force survey). While little research has been 
undertaken into the investment levels needed to ensure a sound HRIS drawing primarily on 
administrative data sources, estimates of the cost of a comprehensive health information system 
including a human resources component range from US$ 0.53 to US$ 2.99 annually per capita 
(24). The cost of a household- or facility-based assessment with a sufficient sample size allowing 
for disaggregated estimates will vary depending on the level of technical support required in the 
country and the final sample size, ranging from US$ 350 000 to over US$ 1 million. In general, 
guidelines suggest that health information, monitoring and evaluation costs comprise between 3% 
and 11% of total project funds (25). 



Given the diversity of potential information sources, monitoring and evaluation of human resources for health 
requires good collaboration between the ministry of health and other sectors that can be reliable sources of 
information, notably the central statistical office, ministry of education, ministry of finance, ministry of labour, 
health professional regulatory and licensing bodies, associations of private providers, and individual health-care 
facilities and health training institutions. Ideally, a commitment should be established in advance to investigate 
purposeful ways to put the data to use. Discussions between representatives of the various stakeholder groups, 
under the leadership of the ministry of health, are recommended from the beginning to set an agenda for data 
harmonization, publication and use, taking into account the timeline for data collection and processing and the 
information needs for health workforce policy and planning. 

Selected tools 

Dal Poz MR, et al. (eds.). Handbook on monitoring and evaluation of human resources for health. Geneva, World 
Health Organization, World Bank and United States Agency for International Development, 2009 (http://www. 
who.int/hrh/resources/handbook/en/index.html, accessed March 27, 2010). 

This Handbook aims to strengthen country technical capacity to accurately monitor their health workforce. 
It offers health managers, researchers and policy-makers a comprehensive and standard reference for 
monitoring and evaluating human resources for health, and brings together an analytical framework with 
strategy options for improving the health workforce information and evidence base, as well as country 
experiences to highlight approaches that have worked. 

Further reading 

Human Resources for Health Action Framework (http://www.capacityproject.org/framework, accessed March 

22 , 2010 ). 

Human Resources for Health (HRH) tools and guidelines. HRH situation analysis. Geneva, World Health 
Organization (http://www.who.int/hrh/tools/situation_analysis/en/index.html, accessed March 22, 2010). 
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Human Resources for Health (HRH) tools and guidelines. HRH planning. Geneva, World Health 
Organization (http://www.who.int/hrh/tools/planning/en/index.html, accessed March 22, 2010). 

Human Resources for Health (HRH) tools and guidelines. HRH health management systems. Geneva, World 
Health Organization (http://www.who.int/hrh/tools/management/en/index.html, accessed March 22, 2010). 

HRIS Strengthening Implementation Toolkit (http://www.capacityproject.org/hris/hris-toolkit/index.html, 
accessed March 22, 2010). 

Human Resources for Health (online journal) (http://www.human-resources-health.com, accessed March 22,2010). 

Spotlight on Health Workforce Statistics (series of fact files). Geneva, World Health Organization (http:// 
www.who.int/hrh/statistics/spotlight/en/index.html, accessed March 22, 2010). 

Service Availability Mapping (SAM) (assessment tool and country reports). Geneva, World Health Organization 
(http://www.who.int/healthinfo/systems/serviceavailabilitymapping/en/, accessed July 26,2010). 
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Annex. Mapping health workforce statistics: Relevant codes in selected 
international standard classifications for social and economic statistics 


Classification of education and training 

Comparability of information on education and training of health workers can be enhanced through the 
collection, processing and dissemination of data following the ISCED, which provides a framework for the 
compilation and presentation of national and international education statistics and indicators for policy analysis 
and decision-making. The ISCED allows a variety of education programmes to be classified by level and field of 
education. Most specializations relevant to health workforce monitoring fall under subfield 72 “health”, including 
education in medicine, medical and health services, nursing and dental services (Table 2.A.1). Depending on 
the field, relevant levels may range from upper secondary to second stage of tertiary education. 


Table 2.A.1 Fields of education related to health in the International Standard Classification of 
Education (ISCED-97) 


Code 

Name 

Specializations 


Fields of education directly related to health 

72 

Health 

Medicine 

The study of the principles and procedures used in preventing, diagnosing, caring 
for and treating illness, disease and injury in humans and the maintenance of 
general health. Principally, this field consists of training of physicians. 



Medical services 

The study of physical disorders, treating diseases and maintaining the physical well¬ 
being of humans, using non-surgical procedures. 



Nursing 

The study of providing health care for the sick, disabled or infirm and assisting 
physicians and other medical and health professionals diagnose and treat patients. 



Dental services 

The study of diagnosing, treating and preventing diseases and abnormalities of the 
teeth and gums. It includes the study of designing, making and repairing dental 
prostheses and orthodontic appliances. It also includes the study of providing 
assistance to dentists. 

Fields of education associated with health 

76 

Social services 

Social work and 
counselling 

The study of the welfare needs of communities, specific groups and individuals and 
the appropriate ways of meeting these needs. Programmes in social work, social 
welfare, crisis support and counselling are included here. 

85 

Environmental 

protection 

Environmental 

protection 

The study of the relationships between living organisms and the environment in 
order to protect a wide range of natural resources. Programmes in services to the 
community dealing with items that affect public health, such as hygiene standards 
in food and water supply, are included here. 

86 

Security services 

Occupational health 
and safety 

The study of recognizing, evaluating and controlling environmental factors 
associated with the workplace. Programmes in occupational health and industrial 
hygiene, labour welfare (safety) and ergonomics are included here. 


Source: adapted from (16, 19). 


Classification of occupations 

To facilitate the harmonization of information on the health workforce situation within and across countries, 
data on health workers should ideally be mapped to the latest revision of ISCO (or its national equivalent). This 
classification offers a system for classifying and aggregating occupational information for purposes of statistical 
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delineation, description and analysis. It uses a hierarchical structure of occupational titles and codes, essentially 
reflecting the distinction of subgroups of the health workforce according to assumed differences in skill level 
and skill specialization required to fulfil the tasks and duties of jobs (Table 2.A.2). 


Table 2.A.2 Occupational titles related to health in the International Standard Classification of 
Occupations, 2008 revision (ISCO-08) 

Group code 

Sub-major Minor 

Unit 

Occupational title 

22 


HEALTH PROFESSIONALS 

221 


Medical doctors 


2211 

Generalist medical practitioners 


2212 

Specialist medical practitioners 

222 


Nursing and midwifery professionals 


2221 

Nursing professionals 


2222 

Midwifery professionals 

223 

2230 

Traditional and complementary medicine professionals 

224 

2240 

Paramedical practitioners 

226 


Other health professionals 


2261 

Dentists 


2262 

Pharmacists 


2263 

Environmental and occupational health and hygiene professionals 


2264 

Physiotherapists 


2265 

Dieticians and nutritionists 


2266 

Audiologists and speech therapists 


2267 

Optometrists and ophthalmic opticians 


2269 

Health professionals not classified elsewhere 

32 


HEALTH ASSOCIATE PROFESSIONALS 

321 


Medical and pharmaceutical technicians 


3211 

Medical imaging and therapeutic eguipment technicians 


3212 

Medical and pathology laboratory technicians 


3213 

Pharmaceutical technicians and assistants 


3214 

Medical and dental prosthetic and related technicians 

322 


Nursing and midwifery associate professionals 


3221 

Nursing associate professionals 


3222 

Midwifery associate professionals 

323 

3230 

Traditional and complementary medicine associate professionals 
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Continued 



Group code 



Sub-major 

Minor 

Unit 

UCCUpdllOrldl 11116 


325 


Other health associate professionals 



3251 

Dental assistants and therapists 



3252 

Medical records and health information technicians 



3253 

Community health workers 



3254 

Dispensing opticians 



3255 

Physiotherapy technicians and assistants 



3256 

Medical assistants 



3257 

Environmental and occupational health inspectors and associates 



3258 

Ambulance workers 



3259 

Health associate professionals not classified elsewhere 

ADDITIONAL HEALTH RELATED UNIT GROUPS 



1342 

Health service managers 



1343 

Aged care service managers 



2634 

Psychologists 



2635 

Social work and counselling professionals 



3344 

Medical secretaries 



5321 

Health-care assistants 



5322 

Home-based personal care workers 



5329 

Personal care workers in health services not classified elsewhere 


Classification of branches of economic activity 

The ISIC classification can form a basis for analysis of data on place of work, as it allows pooling of information 
on workers in health services across different types of economic systems within a comparative framework. In 
ISIC, economic producing units are grouped into successively broader levels of classification in a four-level 
hierarchy; the grouping is done according to similarities in the character of the goods and services produced, 
the uses to which the goods and services are put, and the inputs, process and technology of production. 

Relevant information for health workforce analysis essentially falls under ISIC division 86, “human health 
activities”. Data disaggregated at the lowest levels of classification may allow further analysis across all the 
different types of health systems activities, including public health services, supplies procurement and financing 
(Table 2.A.3). 


Continues... 
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Table 2.A.3 Economic sectors related to health in the International Standard Industrial 
Classification of all Economic Activities (ISIC), fourth revision 


Code 

Section 

Division 

Group 

Class 

tiuiiuiiiii diuvuy 

Core health industry groups and classes 

Q 

86 



Human health activities 



861 

8610 

Hospital activities 



862 

8620 

Medical and dental practice activities 



869 

8690 

Other human health activities 

Selected associated classes 

C 

21 

210 

2100 

Manufacture of pharmaceuticals, medicinal, chemical and botanical products 


32 

325 

3250 

Manufacture of medical and dental instruments and supplies 

E 

36 

360 

3600 

Water collection, treatment and supply 


37 

370 

3700 

Sewerage 

G 

47 

477 

4772 

Retail sale of pharmaceutical and medical goods, cosmetic and toilet articles in specialized stores 

K 

65 

651 

6512 

Non-life insurance (including provision of health insurance) 

M 

71 

712 

7120 

Technical testing and analysis (include testing in the field of food hygiene; testing and measuring 
air and water pollution) 

0 

84 

841 

8412 

Regulation of the activities of providing health care, education, cultural services and other social 
services 




8430 

Compulsory social security activities (including funding and administration of government- 
provided social security programmes for sickness, work-accident, temporary disablement, etc.) 

Q 

87 

871 

8710 

Residential nursing care facilities 



872 

8720 

Residential care activities for mental retardation, mental health and substance abuse 


88 

881 

8810 

Social work activities for the elderly and disabled (without accommodation) 


Source: ( 15 ). 


Other classifications 

It is also of significance to countries and stakeholders to be able to distinguish the different categories of human 
resources within health systems, such as those who are regular employees of the systems and those who are not, 
or those whose basic salaries are drawn from the government budget in comparison with health workers who 
are funded by other sources. A full list of international classifications for the collection and dissemination of 
economic and social statistics is available at the United Nations Statistics Division web site ( 26 ). 
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Health information systems 

3.1 Introduction 

Sound and reliable information is the foundation of decision-making across all health system building blocks. 
It is essential for health system policy development and implementation, governance and regulation, health 
research, human resources development, health education and training, service delivery and financing. 

The health information system provides the underpinnings for decision-making and has four key functions: 
(i) data generation, (ii) compilation, (iii) analysis and synthesis, and (iv) communication and use. The health 
information system collects data from health and other relevant sectors, analyses the data and ensures their 
overall quality, relevance and timeliness, and converts the data into information for health-related decision¬ 
making (1). 

The health information system is sometimes equated with monitoring and evaluation but this is too reductionist 
a perspective. In addition to being essential for monitoring and evaluation, the information system also serves 
broader objectives, such as providing an alert and early warning capability, supporting patient and health 
facility management, enabling planning, underpinning and stimulating research, permitting health situation 
and trends analyses, orienting global reporting, and reinforcing communication of health challenges to diverse 
users. Information is of little value if it is not available in formats that meet the needs of multiple users, i.e. 
policy-makers, planners, managers, health-care providers, communities and individuals. Dissemination and 
communication are therefore essential attributes of the health information system. 

Health planners and decision-makers need different kinds of information including: 

• health determinants (socioeconomic, environmental, behavioural and genetic factors) and the contextual 
environments within which the health system operates); 

• inputs to the health system and related processes (policy and organization, health infrastructure, facilities 
and equipment, costs, human and financial resources and health information systems); 

• the performance or outputs of the health system (availability, accessibility, quality and use of health 
information and services, responsiveness of the system to user needs, and financial risk protection); 

• health outcomes (mortality, morbidity, disease outbreaks, health status, disability and wellbeing); and 

• health inequities (determinants, coverage of use of services, and health outcomes, and including key 
stratifiers such as sex, socioeconomic status, ethnic group and geographical location). 

• A good health information system brings together all relevant partners to ensure that users of health 
information have access to reliable, authoritative, usable, understandable and comparative data. 

3.2 Expectations from country health information systems 

Health information systems serve multiple users and a wide array of purposes that can be summarized as the 
generation of information to enable decision-makers at all levels of the health system to identify problems and 
needs, make evidence-based decisions on health policy and allocate scarce resources optimally (I). Data from 
different sources are used for several purposes at different levels of the health-care system. 
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• Individual level data about the patient’s profile, health-care needs and treatment serve as the basis for 
clinical decision-making. Health-care records provide the basis for sound individual clinical care. Problems 
can arise when health workers are overburdened by excessive data and reporting demands from multiple 
and poorly coordinated subsystems. 

• Health facility level data, both from aggregated facility level records and from administrative sources, such 
as drug procurement records, enable health-care managers to determine resource needs, guide purchasing 
decisions for drugs, equipment and supplies, and develop community outreach. Data from health facilities 
can provide immediate and ongoing information relevant to public health decision-making, but only if 
certain conditions are met. The data must be of high quality, relate to all facilities (public and private), and 
be representative of the services available to the population as a whole. 

• Population level data are essential for public health decision-making and generate information not only 
about those who use the services but also, crucially, about those who do not use them. Household surveys 
have become a primary source of data in developing countries where facility-based statistics are of limited 
quality. Household surveys are needed everywhere, however, because they are the only good source of 
information on individual beliefs, behaviours and practices that are critical determinants of health-care use 
and health status. 

• Public health surveillance brings together information from facilities and communities with a main focus 
on defining problems and providing a timely basis for action. This is especially important when responses 
need to be urgent, as for epidemic diseases. The need for timeliness of reporting and response and the 
requirement for effective linkages, to those in authority with the responsibility for disease control, impose 
additional requirements on health information systems. 

Recognition of the importance of health information systems to be capable of generating reliable data is growing. 
In many countries, health sector reform and decentralization have brought about shifts in functions between 
the central and peripheral levels and have generated new information needs with changing requirements 
for data collection, processing, analysis and dissemination. Health sector reforms also magnify the need for 
standardization and quality of information. 

Performance and results based monitoring, stimulated by unprecedented increases in development assistance 
and global health initiatives — such as the Global Alliance on Vaccines Initiative (GAVI), Global Fund to 
Fight AIDS, Tuberculosis and Malaria (GFATM), the United States President’s Emergency Plan for AIDS Relief 
(PEPFAR), and the Roll Back Malaria partnership — have increased pressure on governments and organizations 
to improve their performance and demonstrate tangible results to their stakeholders. In this environment, a 
premium has been placed on the existence of adequate health information of good quality. Health information 
systems are called upon to enable tracking along the continuum of inputs to the health system, processes and 
outputs, as well as outcomes and impact. 

Few developing countries have sufficiently strong and effective health information systems to meet all these 
diverse information needs. New technologies can contribute to improving data generation, compilation and 
exchange but will require the existence of clear data quality standards to be of optimal value. 

3.3 Sources of information on country health information systems 

Information about the functioning of the health information system can be obtained from the different sectors 
and agencies that are responsible for the generation, synthesis, analysis and use of data at the country, regional 
and global levels. At the country level, the ministries of health record the timeliness and quality of data reported 
through health services and disease surveillance systems. National Statistics Offices maintain information on 
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the availability and quality of data generated through major data collection undertakings, such as the decennial 
census, large-scale household surveys and the civil registration system. As custodians of national official 
statistics, they often have explicit requirements for the method of data collection, compilation and sharing, 
and adhere to the Fundamental principles of official statistics devised by the United Nations (2). International 
agencies working in health also maintain information about the availability and quality of data on international 
health goals, including (but not limited to) the MDGs. 

3.4 Criteria for assessing country health information systems performance 

Criteria for assessing performance of health information systems and the quality of data they generate have 
rarely been defined, let alone implemented. In contrast, in sectors other than health — particularly for 
macroeconomic and financial statistics — considerable work has been done to define standards, guidelines and 
best practices (see Box 3.1). 



Box 3.1 Tools to assess the quality of national statistics 

The Organisation for Economic Co-operation and Development (OECD) has developed statistical 
standards, guidelines and best practices on development indicators (3). Explanatory metadata are 
collected and published to accompany all data thus enabling users to assess data quality, i.e. fitness 
for use. Based on these metadata, comparability across countries — an important aspect of data 
quality — can be assessed. 

Some national statistics offices have commissioned external reviews of performance to identify 
strengths and weaknesses and to make recommendations for improvement (4). The United 
Nations Fundamental principles of official statistics is often used as a general framework to assess 
the performance of national statistics offices (5). The quality of information is central to its 
usefulness. Information must be reliable, up to date, independent and trustworthy. At the same 
time, it is important to avoid duplication of efforts and minimize the burden of data collection on 
front-line staff, so that data are generated with minimal disruption of the delivery of care. 

Although there is wide agreement in the literature on what the components of data quality should 
be, there is no universal consensus on how to group them. Some authors have proposed headings 
covering accuracy, relevance, coherence and consistency, continuity, timeliness, accessibility 
and revisability (6). Others include coherence (especially comparability), availability and clarity 
(7). Clearly, conflicts can arise between different facets, such as those between consistency and 
timeliness, and trade-offs must be made. The extent to which statistics meet users’ needs and 
expectations for statistical information is widely recognized to be of paramount importance. To 
allow users to assess the quality of the statistics they utilize, producers of statistics provide neutral, 
descriptive information about all aspects of statistics that affect users’ views on how well the 
statistics might meet their needs and expectations. 
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The International Monetary Fund (IMF) has developed the General Data Dissemination Strategy 
(GDDS) to help countries in improving the quality of their data (8). The GDDS strategy provides a 
framework for evaluating needs for data improvement and setting priorities; it provides guidance 
on dissemination to the public of comprehensive, timely, accessible and reliable economic, financial 
and sociodemographic statistics. 

Arising out of the dissemination strategy, the IMF has produced a Data Quality Assessment 
Framework (DQAF), which identifies quality-related features of governance of statistical systems, 
processes and products. The framework is rooted in the Fundamental principles of official statistics 
and describes five dimensions of quality — assurances of integrity, methodological soundness, 
accuracy and reliability, serviceability and accessibility. The framework, which is used for 
comprehensive assessments of countries’ data quality, covers institutional environments, statistical 
processes and characteristics of the statistical products (9). 

The Food and Agriculture Organization (FAO) has developed a Data Quality Stamp for statistical 
data that meet quality criteria including: availability of appropriate metadata for all data series; use 
of international classifications; provision of an updated schedule to ensure timeliness; provision 
of global coverage information in the data series; integration of the data series in the databases 
within a statistical framework; and assurance that the data series is up to date (10). 


A commonly used standards framework is provided by the Fundamental principles of official statistics (2). Data 
quality assurance approaches generally distinguish assessment criteria for data outputs from those that relate to 
the quality of institutional frameworks, which is a prerequisite for the generation of reliable data. Some of the 
quality frameworks are intended to be used to assess national level data, whereas others relate to the quality of 
data issued by international agencies, such as the World Bank or the International Monetary Fund (IMF). 

The Health Metrics Network (HMN) framework identifies the key components and standards of a country 
health information system (1). The framework describes health information system components in terms of 
resources, indicators, data sources, data management, information products, and dissemination and use and 
specifies the standards to be attained for each component. 

3.5 Methods for assessing country health information system performance 

A country’s health information system performance can be assessed either by using a self-administered tool or 
through independent (often external) expert evaluation (see the matrix in Table 3.1). The major advantage of 
self-assessment approaches is the degree of country ownership generated that enables the assessment to serve as 
the basis for the development of a plan for improvement. However, self-assessment approaches are generally 
time consuming and complex to implement; they are less likely to generate results that can be compared over 
time or between countries and are more likely to be biased. Independent assessment is generally based on existing 
sources, such as databases of international agencies, so as to minimize the reporting burden on countries. The 
disadvantage is that countries may not agree with the assessment and therefore may not use the results for 
developing an improvement strategy. 
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Table 3.1 Assessment matrix for country information systems performance 


National data process 

Self-assessment 

Independent assessment 

Statistical system 

Health information system 

General Data Dissemination Strategy 

Health Metrics Network self-assessment 
tool 

World Bank Statistical Capacity Indicator score 

Health Information System Performance Index (HISPIX) 

Specific indicators: reporting rates, data and statistics availability 


Self-assessment approaches 

The General Data Dissemination Strategy (GDDS) developed by IMF (8) is designed: 

• to assist countries in assessing and documenting their statistical practices and procedures and in compiling 
metadata; 

• to enable countries to develop and implement plans for improvement in the different areas of statistics 
covered by the GDDS. 

The principal goal of the GDDS is to improve data quality — quality relating both to the actual data and also 
to the whole statistical system. The strategy involves providing short-term technical assistance to countries to 
engage in a systematic review of existing statistics as compared with international standards — essentially an 
externally facilitated self-assessment. The process engages both data producers and data users and seeks to bring 
about more effective communication both among national statistical agencies and with the user community. 
Based on the result of the assessment, countries develop a comprehensive improvement plan for the statistical 
system. The GDDS has been used as the basis for the formulation of national strategies for the development of 
statistics. 

The HMN assessment tool brings together country users and producers of health data to assess the strengths 
and weaknesses of the national health information system. Like the GDDS, it involves a facilitated assessment 
that is intended to guide countries’ efforts to strengthen their health information systems by enabling a baseline 
analysis and identification of areas for improvement in which donor support might be sought. The HMN 
assessment tool framework follows a cascading structure that flows from five main dimensions of data quality: 
(i) integrity, (ii) methodological soundness, (iii) accuracy and reliability, (iv) serviceability, and (v) accessibility. 
For each of these interrelated dimensions, the framework identifies pointers, or observable features, that can 
be used in assessing quality. In addition to these five dimensions, the tool describes a set of prerequisites for 
the assessment of data quality. The coverage of these dimensions recognizes that data quality encompasses 
characteristics related to the institution or system used for the production of the data as well as characteristics 
of the individual data product. By engaging all stakeholders, it helps develop a shared vision of a more coherent, 
integrated, efficient and useful system. 

More than 50 countries completed their health information systems assessment by the end of 2009 (11). 
Overall, the self-assessment approach appears to have worked well in generating a broad understanding of the 
HMN concept of health information systems, cutting across both disease-based and source-based reserves of 
information. This tool resulted in enhanced collaboration among various stakeholders in health information, 
particularly between health and statistics constituencies. However, considering the degree of stakeholder 
involvement required, the approach is complex and time consuming to administer. There are issues of 
objectivity of the respondents and possible conflicts of interest that arise when a self-assessment approach 
used is not well-suited to enabling comparisons between countries and over time. 
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Independent assessments 

The World Bank Statistical Capacity Indicator is calculated on the basis of a desk review by external technical 
experts. This summary measure provides an overview of the statistical capacity of developing countries and 
is based on a diagnostic framework developed with a view to assessing the capacity of statistical systems 
using metadata information generally available for most countries. The framework has three dimensions: (i) 
statistical practice (the ability to adhere to internationally recommended standards and methods); (ii) data 
collection (frequency of censuses/surveys and completeness of vital registration); and (iii) indicator availability 
(availability and frequency of key socioeconomic indicators). Countries are scored against specific criteria, using 
information available from the World Bank, IMF, the United Nations, United Nations Educational, Scientific 
and Cultural Organization (UNESCO) and WHO. A composite score for each dimension and an overall score 
combining all three dimensions is derived for each country on a scale of 0-100. A score of 100 indicates that 
the country meets all the criteria and suggests good statistical standing. The assessment is carried out annually. 

3.6 Core indicators 

Indicators of country health information system performance can be categorized as two broad types: 

1. Indicators related to data generation using core sources and methods (health surveys, civil registration, 
census, facility reporting, health system resource tracking). These reflect country capacity to collect relevant 
data at appropriate intervals and uses the most appropriate data sources. Benchmarks include periodicity, 
timeliness, contents of data collection tools and availability of data on key indicators. 

2. Indicators related to country capacities for synthesis, analysis and validation of data. These measure 
key dimensions of the institutional frameworks needed to ensure data quality, including independence, 
transparency and access. Benchmarks include the availability of independent coordination mechanisms 
and the availability of microdata and metadata. 

The following indicators are grouped according to the assessed data sources that make up a country’s health 
information system. The last group of indicators addresses the capacity for data synthesis, analysis and validation. 
Table 3.2 defines the indicators listed below and describes methods of measurement. 

Recommended indicators to assess data sources: 

Health surveys 

• Country has a 10-year costed survey plan that covers all priority health topics and takes into account other 
relevant data sources. 

• Two or more data points available for child mortality in the past five years. 1 

• Two or more population-based data points for maternal mortality in the past 10 years, including one in the 
past five years. 

• Two or more data points for coverage of key health interventions in the past five years. 

• One or more data point on smoking and adult nutritional status in the past five years. 


1 Only relevant to countries without complete civil registration systems (>90% coverage of births and death). 
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Birth and death registration: 

• Birth registration of at least 90% of all births (intermediate goal 50%). Indicator: percentage of births 
registered. 

• Death registration of at least 90% of all deaths (intermediate goal 50%). Indicator: percentage of deaths 
registered. 

• ICD-10 used in district hospitals and causes of death reported to national level. 

Censuses 

• Census completed within the past 10 years. 

• Population projections for districts and smaller administrative areas available for the next 10 years, in print 
and electronically, and well documented. 

Health facility reporting 

• Number of institutional deliveries available, by district, and published within 12 months of the preceding year. 

• HIV prevalence for relevant surveillance populations published within 12 months of the preceding year. 

• Country web site for health statistics, with latest report and data available to the general public. 

• Reporting of notifiable diseases makes use of modern communication technology, and reporting of 
statistics from district to national levels is web-based. 

• At least 90% of the districts submit timely, complete, accurate reports to national level. Indicator: 
percentage of districts that submit timely, complete, accurate reports to national level. 

• Data quality assessments carried out and published within the past three years, using internationally agreed 
quality criteria, such as Data Quality Assessment Framework (DQAF). 

• International Health Regulations implemented according to international standards. 

Health system resource tracking 

• At least one national health accounts exercise completed in the past five years. 

• National database with public and private sector health facilities and geocoding, available and updated 
within the past three years. 

• National database with health workers by district and main cadres updated within the past two years. 

• Annual data on availability of tracer medicines and commodities in public and private health facilities. 

Recommended indicators to assess capacity for analysis, synthesis and validation of 
health data 

• A designated and functioning institutional mechanism charged with analysis of health statistics, synthesis of 
data from different sources and validation of data from population-based and facility-based sources. 

• A national set of indicators with targets and annual reporting to inform annual health sector reviews and 
other planning cycles. 

• A national microdata archive for health surveys and census that is established and operational. 

• Survey data used to assess and adjust routine reports from health facility on vaccinations, with the results 
published within 12 months of the preceding year. 

• A burden of disease study conducted within the past five years, with a strong national contribution. 

• A health systems performance assessment carried out within the past five years, with a strong national 
contribution. 
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Table 3.2 Summary of core indicators and scoring for Health Information Systems Performance 
Index (HISPIX) 


Indicators 

Definition 

Data collection 
method 

Scoring for HISPIX 

Health surveys 

Country has a 10-year costed survey plan that covers 

Survey plan comprises modular contents with 

Bureau of the 

Yes: 1 

all priority health topics and takes into account other 

periodicity for specific indicators calibrated to achieve 

Census, National 


relevant data sources. 

maximum sensitivity and efficiency. Includes data 

Statistics Office and 

No: 0 


collection concerning health-related behaviours and 
bio-clinical measurements 

Ministry of Health 


Two or more data points available for child mortality 


Country reports, 

Yes: 1 

in the past five years 3 


DHS b and MICS C 

No: 0 

Two or more population-based data points for 


Country reports, 

Yes: 1 

maternal mortality in the past 10 years, including 
one in the past five years 3 


DHS and MICS 

No: 0 

Two or more data points for coverage of key health 

Comprising coverage of key maternal and child health- 

Country reports, 

Yes: 1 

interventions in the past five years 

care interventions, risk behaviours and care-seeking 

DHS and MICS 

No: 0 

One or more data points on smoking and adult 

Nutritional status clinically measured 

Country reports, 

Yes: 1 

nutritional status in the past five years 


DHS and MICS 

No: 0 

Birth and death registration 

Birth registration of at least 90% of all births 

Numerator: number of births registered, as reported 

Civil registration or 

<50% score 0 

(intermediate goal 50%). Indicator: percentage of 

by civil or sample registration systems, hospitals and 

sample registration 


births registered 

community-based reporting systems 

systems 

50-89% score 1 


Denominator : total births for the same time period and 
geographical region. Where information on total births 
is not available because of incomplete civil registration, 
total births can be estimated by extrapolating from the 
census or on the basis of information about natality 
rates derived from population surveys 


>90% score 2 

Death registration of at least 90% of all deaths 

Numerator: number of deaths registered as reported 

Civil or sample 

<50% score 0 

(intermediate goal 50%). Indicator: percentage of 

by civil or sample registration systems, hospitals and 

registration systems 


deaths registered 

community-based reporting systems 


50-89% score 1 


Denominator: total deaths for the same time period and 
geographical region. Where information on total deaths 
is not available because of incomplete civil registration, 
total deaths can be estimated by extrapolating from the 
census or on the basis of information about mortality 
rates derived from population surveys 


>90% score 2 

ICD-10 d used in district hospitals and causes of 

Numerator: number of district hospitals using ICD-10 to 

Routine Health 

<50% score 0 

death reported to national level 

certify cause of death 

Management 
Information System 

50-89% score 1 


Denominator: total district hospitals 

HM IS reports 

>90% score 2 
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Continued 


Indicators 

Definition 

Data collection 
method 

Scoring for HISPIX 

Census 

Census completed within the past 10 years 

Population projections for districts and smaller 
administrative areas available for next 10 years, in 
print and electronically, well documented 


Bureau of the 

Census, National 
Statistics Office and 
Ministry of Health 

1 

1 

Health facility reporting 

Number of institutional deliveries available, by 
district, and published within 12 months of 
preceding year 

Includes deliveries in public, private and 
nongovernmental organization facilities 

Country H MIS 
reports 

1 

HiVprevalence for relevant surveillance populations 
published within 12 months of preceding year 


National Aids 
Committee reports 

1 

Country web site for health statistics, with latest 
report and data available to the general public 


Country HIS e reports 

1 

Reporting of notifiable diseases makes use of 
modern communication technology, and reporting 
of statistics from district to national levels is 
web-based 


Country H MIS 
reports 

1 

At least 90% of districts submit timely, complete, 
accurate reports to national level. Indicator: 
percentage of districts that submit timely, complete, 
accurate reports to national level 

Numerator: number of health districts with timely and 
complete reporting of key data series 

Denominator: total districts 

Country H MIS 
reports 

1 


Countries should define core data series that should be 
reported to districts by all facilities and compare reports 
against this list 



Data quality assessments carried out and published 
within the past three years, using internationally 
agreed guality criteria such as the Data Quality 
Assessment Framework (DQAF) 

Assessment should routinely cover all administrative 
data sources (e.g. civil registration, facility reports) 

Assessment should use internationally agreed data 
guality criteria such as DQAF 

Country H MIS 
reports 

1 

International Health Regulations (IHR) implemented 
according to international standards 

Compliant with IHR monitoring and evaluation 
framework 

Country health 
sector reports 

1 

Health system resource tracking 

At least one national health accounts exercise 
completed in the past five years 


NHA f report 

1 

National database with public and private sector 
health facilities and geocoding, available and 
updated within the past three years 

Database should separate public, private and non¬ 
profit facilities; it should also include key infrastructure, 
human resources, medicines, eguipment and supplies, 
and service availability 

Health facility 
assessments 

1 

National database with health workers by district 
and main cadres updated within the past two years 

Database comprises data from multiple sources, 
including census, labour force surveys, professional 
registers, training institutions and facility assessments 

National health 

sector review 

1 

Annual data on availability of tracer medicines and 
commodities in public and private health facilities 

Aligned to national essential medicines list 

Essential medicines 
reviews; health 
facility assessments 

1 
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Indicators 

Definition 

Data collection 
method 

Scoring for HISPIX 

Capacity for analysis, synthesis and validation of health data 

A designated and functioning institutional mechanism 
charged with analysis of health statistics, synthesis 
of data from different sources and validation of data 
from population-based and facility-based sources 

Body should be administratively separate from 
programmes responsible for delivery of interventions. 
Should adhere to Fundamental principles of official 
statistics 

National health 
sector reports 

1 

A national set of indicators with targets and annual 
reporting to inform annual health sector reviews and 
other planning cycles 

Indicators cover key issues including health 
determinants, health system inputs, processes 
and outputs, use of health care services, mortality, 
morbidity, health system responsiveness, etc. 

National health 
sector reports 

1 

A national microdata archive for health surveys and 
census established and operational 



1 

Survey data used to assess and adjust routine reports 
from health facility on vaccinations, with the results 
published within 12 months of the preceding year 

Validation by an independent reviewer would be 
needed to ascertain the extent of analysis and 
validation 

Information 
available from 
health statistics 
reports 

1 

A burden of disease study conducted within the past 
five years, with a strong national contribution 



1 

A health systems performance assessment carried 
out within the past five years, with a strong national 
contribution 



1 

Overall HISPIX 
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“Only relevant to countries without complete civil registration systems (>90% coverage of births and death). 
b Demographic and Health Surveys. 

'Multiple Indicator Cluster Surveys. 

d International statistical classification of diseases and related health problems, 10th revision (iCD-10). 2nd edition. Geneva, World Health 
Organization, 2005 (http://www.who.int/classifications/icd/en/). 
e Health Information System. 
f National Health Accounts. 

3.7 Summary measure of health information system performance 

WHO is proposing a Health Information System Performance Index (HISPIX) — a summary measure based 
on the above-mentioned standardized indicators for assessing data quality and the overall performance of 
the health information system. The score is calculated from information available in the public domain using 
standard indicators to enhance objectivity and comparability over time and across countries. 

For the majority of the indicators, a simple binary scoring system (“yes” or “no”) is used, with no weighting. For 
the few indicators that are measured in terms of percentages, the score is calculated as described in Table 3.2. 
The advantage of this approach is that it permits countries and development partners to identify key areas for 
improvement as part of a health information system strengthening plan. 

The crucial difference between the HISPIX approach and the HMN self-assessment tool is that the indicators 
can be assessed on the basis of information that is largely available in the public domain. Information on data 
sources and data availability can be compiled from WHO databases and those of other international agencies. 
Information on inputs and resources is available from country health statistics reports and from the self- 
assessments conducted though HMN. For countries that have not conducted such assessments, it may be 
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necessary to gather information through WHO and other agency country offices and through direct contacts 
with country health information units and statistics offices. 

Because of the relative ease of data collection, and because several of the indicators are amenable to relatively 
rapid change, it is suggested that the analysis be conducted every three years to gauge trends over time. 

Efforts should continue, however, to link the independent assessment approach to self-assessment strategies, 
such as the HMN tool. This would help ensure continued country involvement and thus sustainability. Clear 
descriptions of data quality criteria and user-friendly checklists could be valuable tools in this regard. 

Selected resources and tools 

The following annotated references and links to key sources of standards, guidelines and quality criteria for 
different components of the health information system are not necessarily assessment tools in themselves; they 
offer guidance on quality criteria for various aspects of the health information system. The main focus is on 
ensuring data quality. 

General health information systems assessment 

Health Metrics Network. Assessing the national health information system: an assessment tool, version 4.00. 
Geneva, World Health Organization, 2008 (http://www.who.int/healthmetrics/tools/hisassessment/en/index. 
html, accessed 1 April 2010). 

This is a tool for conducting a systematic assessment of the existing national health information systems 
— both to establish a baseline and to monitor progress. The assessment is aligned with the standards for 
health information systems described in the HMN framework. The assessment covers the many subsystems 
of a national health information system, including public and private sources of health-related data. It 
addresses the resources available to the system (inputs), its methods of work and products (processes 
and outputs) and results in terms of data availability, quality and use (outcomes). All major stakeholders 
should participate in assessing the national health information system and planning for its strengthening. 
Stakeholders include the producers, users and financiers of health information and other social statistics 
at various national and sub-national levels: officials in government ministries and agencies; donors and 
development partners such as multilateral and bilateral agencies; nongovernmental organizations; academic 
institutions; professional associations; other users of health-related information such as parliamentarians; 
civil society (including health-related advocacy groups); and the media. The tool is also available as an 
electronic spreadsheet to facilitate scoring of the indicators. 

Health surveys 

United Nations Statistics Division. Household sample surveys in developing and transition countries. New York, 
NY, United Nations, 2005 (Series F, No. 96, sales number E.05.XVII.6; http://unstats.un.org/unsd/hhsurveys/, 
accessed 1 April 2010). 

This handbook describes standards for household surveys but is not an assessment tool. The publication 
presents the “state of the art” on important aspects of conducting household surveys in developing and 
transition countries, including sample design, survey implementation, non-sampling errors, survey costs, 
and analysis of survey data. The main objective of this handbook is to assist national survey statisticians 
to design household surveys in an efficient and reliable manner, and to allow users to make greater use of 
survey generated data. 
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MEASURE DHS. Demographic and health surveys (DHS). (http://www.measuredhs.com/, accessed 1 April 

2010 ). 

The USAID-supported surveys, implemented by ICF Macro, are based on specified standards and 
quality criteria. DHS has been conducted around the world for more than two decades. These nationally 
representative household surveys include large sample sizes (usually between 5000 and 30 000 households) 
and are generally conducted every five years to allow for comparisons over time. DHS staff members have 
advanced training in economics, sociology, behavioural psychology, statistics, management and social 
marketing. 

Civil registration 

United Nations Statistics Division. Principles and recommendations for a vital statistics system, revision 2. New 
York, NY, United Nations, 1998 (Series M, No. 19/Rev. 2, sales number 01.XVII.10). 

This serves as a guide for national governments in establishing and maintaining reliable civil registration 
systems for legal documentation on events throughout the lifetime of individuals: birth, changes in 
marital status, and death. This book provides technical guidance on standards, concepts, definitions and 
classifications for civil registration and vital statistics to further increase the international comparability of 
data. Companion publications include the handbooks on civil registration and vital statistics systems issued by 
the United Nations over the past fewyears (http://unstats.un.org/unsd/demographic/standmeth/handbooks/ 
default.htm#civilreg). 

Censuses 

United Nations Statistics Division. Principles and recommendations for population and housing censuses (2008). 
New York, NY, United Nations, 1998 (http://unstats.un.org/unsd/demographic/sources/census/census3.htm, 
accessed 1 April 2010). 

This provides international principles and recommendations for use by national statistical offices and 
census officials throughout the world in planning and organizing a census. 

Health facility reporting 

Lafond A, Field R. The PRISM: introducing an analytical framework for understanding performance of routine 
health information systems in developing countries. Presented at: Workshop on Enhancing the Quality and Use 
of Health Information at the District Level, Eastern Cape Province, South Africa, 29 September-4 October 2003 
(http://www.globalhealthcommunication.org/tool_docs/64/the_prism_introducing_an_analytic_framework_ 
for_undersanding.pdf, accessed 1 April 2010). 

PRISM (Performance of Routine Information System Management), has been developed by MEASURE 
Evaluation and John Snow, Inc. It is designed to assess routine, facility-based information and management 
systems while acknowledging the broader context in which such systems operate. It emphasizes strengthening 
routine health information system performance through better data quality and improved information 
use. PRISM broadens the analysis of performance to include three key categories of determinants that 
affect performance: 

• behavioural determinants: the knowledge, skills, attitudes, values and motivation of the people who 
collect and use data; 

• technical determinants: data collection forms, processes, systems and methods; 
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• organizational determinants: information culture, structure and resources and the roles and 
responsibilities of key contributors at each level of the health system. 

Health system resource tracking 

WHO/World Bank/USAID. Guide to producing national health accounts with special applications for low- 
income and middle-income countries. Geneva, World Health Organization, 2003 (ISBN 92 4 154607 7, NLM 
classification W 74.1; http://www.who.int/nha/docs/English_PG.pdf, accessed 1 April 2010). 

This document guides the reader through the process of acquiring and evaluating data and provides 
step-by-step examples of how to convert raw numbers into information useful for policy analysis and 
development. 

Assessment of data quality in national statistical systems 

De Vries W. How are we doing? Performance indicators for national statistical systems. Washington, DC, 
International Monetary Fund, 1998 (http://dsbb.imf.org/vgn/images/pdfs/nld.pdf, accessed 1 April 2010). 

This proposes a systems approach for evaluating the performance of national statistical offices and takes 
the view that there is a high correlation between the quality of a statistical system and the quality of its 
products. The United Nations Fundamental principles of official statistics is used as a general framework to 
assess the performance of national statistical offices; the guide provides a brief explanation of the principles 
and raises several operational questions related to each one. 

Elvers E, Rosen B. Quality concept for official statistics. In: Encyclopedia of statistical sciences. Malden, MA, 
John Wiley & Sons, Inc., 1997 (Wiley InterScience Publication, http://dsbb.imf.org/vgn/images/pdfs/Encyc. 
pdf, accessed 1 April 2010). 

Quality of statistics is defined by how well statistics meets users’ needs and expectations for statistical 
information, once disseminated. The authors suggest that producers of official statistics should provide 
neutral, descriptive information about all aspects of statistics, so that users may assess their quality. They 
suggest the information be organized by main quality components, such as content, accuracy, timeliness, 
coherence (especially comparability), availability and clarity. Definitions of these components and their 
subcomponents are provided. 

General data dissemination system (GDDS). Washington, DC, International Monetary Fund, 1995 (http://dsbb. 
imf.org/Applications/web/getpage/?pagename=gddswhatgdds, accessed 1 April 2010). 

The GDDS framework is built around four dimensions — data characteristics, quality, access, and integrity 
— and is intended to provide guidance for the overall development of macroeconomic, financial, and socio¬ 
demographic data. The framework takes into account, across a broad range of countries, the diversity of 
their economies and the developmental requirements of many of their statistical systems. 

Data quality assessment framework (DQAF). Washington, DC, International Monetary Fund, 2003 (http://dsbb. 
imf.org/vgn/images/pdfs/dqrs_factsheet.pdf; http://dsbb.imf.org/vgn/images/pdfs/dqrs_Genframework.pdf, 
accessed 1 April 2010). 

Five dimensions of data quality: assurances of integrity, methodological soundness, accuracy and reliability, 
serviceability, and accessibility, and a set of prerequisites for data quality are the centre of the IMF DQAF 
The DQAF, which is used for comprehensive assessments of countries’ data quality, covers institutional 
environments, statistical processes, and characteristics of the statistical products. 
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Fundamental principles of official statistics. New York, NY, United Nations, 1994 (http://unstats.un.org/unsd/ 
methods/statorg/FP-English.htm, accessed 1 April 2010). 

These were adopted by the Statistical Commission in 1994. While not an assessment tool, the principles 
provide a general quality framework for national statistics offices to review performance, identify strengths 
and weaknesses, and make recommendations for improvement. The ten principles cover: relevance 
impartiality and equal access; professional standards and ethics; accountability and transparency; 
prevention of misuse; cost-effectiveness; confidentiality; legislation; national coordination; international 
standards; and international cooperation. 

Implementation of the fundamental principles of official statistics. Report of the Secretary-General to the Thirty- 
fifth session of the United Nations Statistical Commission, 2-5 March 2004 (http://unstats.un.org/unsd/ 
statcom/doc04/2004-21e.pdf, accessed 1 April 2010). 

The Statistical Commission has developed a questionnaire allowing national statistical offices to report 
their experiences with the fundamental principles in a uniform way. This report presents the main results 
of the survey on the implementation of the Fundamental Principles of Official Statistics conducted by the 
Division between May and November 2003. 

Statistical capacity improvement in IDA countries: progress report. Washington, DC, The World Bank, 2006 
(http://siteresources.worldbank.org/SCBINTRANET/Resources/Statistical_Capacity_Improvement_in_IDA_ 
Countries-Mayl6_2006.pdf, accessed 1 April 2010). 

This paper is the first progress report on statistical capacity improvement in IDA countries. It reports on 
improvements made by IDA member countries to their capacity to produce good quality official statistics. 
It also reviews the support given to countries by the World Bank and by other development partners, 
discusses key issues and constraints, and outlines plans for further work. 

Assessment of data quality in international agencies 

Quality dimensions, core values for OECD statistics and procedures for planning and evaluating statistical activities. 
Paris, Organisation for Economic Co-operation and Development, 2003 (STD/QFS(2003)1; http://www.oecd. 
org/dataoecd/26/38/21687665.pdf, accessed 1 April 2010). 

The guidelines aim to ensure that OECD published statistics meet specific quality criteria. Quality is 
defined as “fitness for use” in terms of user needs. Given the work already done by several statistical 
organisations, the OECD drew on their experience and adapted it to the Organisation’s context. The 
OECD views quality in terms of seven dimensions: relevance; accuracy; credibility; timeliness; 
accessibility; interpretability; and coherence, which are described in this document. 
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4. Access to essential medicines 



* 



Access to essential medicines 

4.1 Introduction 

According to the WHO framework for health systems (1), a well-functioning health system ensures equitable 
access to essential medical products, vaccines and technologies of assured quality, safety, efficacy and cost- 
effectiveness, and their scientifically sound and cost-effective use. To achieve these objectives, the following are 
needed: 

• national policies, standards, guidelines and regulations that support policy; 

• information on prices, the status of international trade agreements and the capacity to set and negotiate 
prices; 

• reliable manufacturing practices when they exist in-country and quality assessment of priority products; 

• procurement, supply and storage, and distribution systems that minimize leakage and other waste; and 

• support for rational use of medicines, commodities and equipment, through guidelines and strategies to 
assure adherence, reduce resistance, maximize patient safety and training. 

Monitoring access to essential medicines is closely intertwined with at least two other building blocks: service 
delivery and governance. Health service delivery is discussed in Section 1 of this handbook while issues related 
to governance are dealt with in Section 6. 

This section of the handbook focuses on essential medicines, i.e. those that satisfy the priority health care needs 
of the population. Essential medicines are intended to be available within the context of functioning health 
systems at all times, in adequate amounts, in the appropriate dosage, with assured quality, and at a price that 
individuals and the community can afford (2). 

Access to medicines is included in the Millennium Development Goals under MDG 8, and specifically Target 
8.E: In cooperation with pharmaceutical companies, provide access to affordable essential drugs in developing 
countries . 1 Access has been defined as “having medicines continuously available and affordable at public or 
private health facilities or medicine outlets that are within one hour’s walk of the population” (3). Given its 
complexity, an overall picture of access to medicines can only be generated using a range of indicators that 
provide data on medicine availability and price, in both public and private sectors, in combination with key 
policy indicators. Recent United Nations reports, that assessed progress towards MDG target 8.E, found that 
low availability, high prices and poor affordability of medicines are key impediments to treatment access in 
low- and middle-income countries (4,5). 


1 Official List of Millennium Development Indicators, effective 15 January 2008. http://mdgs.un.org/unsd/mdg/Host. 
aspx?Content=Indicators/OfficialList.htm, accessed 15 April 2010. 
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4.2 Sources of information on access to essential medicines 


Facility surveys 

A general facility survey usually focuses on a wide range of key health services and collects information on 
facility infrastructure, equipment and supplies, support systems, management systems and providers’ adherence 
to standards. The collection of data on the availability of essential medicines and commodities and on the use of 
these medicines is an essential part of such surveys. 

WHO and Health Action International (HAI) have developed a standardized methodology 2 for facility-based 
surveys of medicine prices, availability, affordability and price components (6, 7). In the survey, data on the 
availability and price of approximately 50 medicines is collected through visits to medicines outlets in the 
public sector, private sector and any other sectors that serve as important medicine dispensing points (e.g. 
NGOs, mission hospitals). The list of survey medicines includes 14 medicines in use worldwide (Table 4.1) 
and 16 regionally specific medicines. In addition, countries are encouraged to collect data on a further 20 
medicines of national importance. For each medicine, data are collected on the originator brand and the lowest- 
priced generic equivalent found at each medicine outlet. Government procurement prices are also collected, 
as are the add-on costs that are charged to medicines as they proceed through the supply and distribution 
chain. The survey is conducted by trained data collectors, following which data is double-entered into a pre¬ 
programmed Excel workbook that allows for standardized analysis. Treatment affordability is estimated by 
comparing medicine costs to the daily wage of the lowest-paid unskilled government worker. 


Table 4.1 Global core list of medicines included in WHO/HAI surveys 



Indication 

Medicine name* 

Strength 

Dosage form 

1 

Asthma 

Salbutamol 

0.1 mg/dose 

inhaler 

2 

Diabetes 

Glibenclamide 

5 mg 

capsule/tablet 

3 

Cardiovascular disease 

Atenolol 

50 mg 

capule/tablet 

4 

Cardiovascular disease 

Captopril 

25 mg 

capsule/tablet 

5 

Cardiovascular disease 

Simvastatin 

20 mg 

capsule/tablet 

6 

Depression 

Amitriptyline 

25 mg 

capsule/tablet 

7 

Infectious disease 

Ciprofloxacin 

500 mg 

capsule/tablet 

8 

Infectious disease 

Co-trimoxazole 

8+40 mg/ml 

suspension 

9 

Infectious disease 

Amoxicillin 

500 mg 

capsule/tablet 

10 

Infectious disease 

Ceftriaxone 

1 g/vial 

injection 

11 

Central nervous system diseases 

Diazepam 

5 mg 

capsule/tablet 

12 

Pain/inflammation 

Diclofenac 

50 mg 

capsule/tablet 

13 

Pain/inflammation 

Paracetamol 

24 mg/ml 

suspension 

14 

Ulcer 

Omeprazole 

20 mg 

capsule/tablet 


“Medicine names may be spelt differently in different countries. 


2 A detailed description of the methodology, as well as country-specific data and reports, can be found at http://www.haiweb.org/ 
medicineprices/, accessed 23 May 2010. 
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Integration of a list of tracer medicines in larger facility censuses or surveys would be desirable. This would allow 
more regular monitoring and integration with other data on health services, such as infrastructure and human 
resources. A list of medicines has been proposed by various disease programmes in WHO for inclusion in the 
service availability and readiness assessment methodology (see Annex to this section) (8). It is recommended 
that all surveys collect, at a minimum, data on the global list of 14 medicines included in WHO/HAI surveys. 
The inclusion of additional medicines should be based on national treatment guidelines, local disease patterns 
and other priorities. Where possible, the use of the WHO/HAI methods of collecting, entering and analysing 
data are strongly encouraged. 

In addition to the availability and price of medicines, it is also important to assess the quality of use aspects, such 
as appropriate prescription practices, rational medicine use and user adherence. This requires a more extensive 
assessment of practices in facilities, including record reviews, exit interviews and observation of patients and 
providers. There are three categories of indicators for which data are collected (9): 

1. prescribing indicators (average number of medicines prescribed per encounter, percentage of medicines 
prescribed by a generic name, percentage of encounters with an antibiotic prescribed, percentage of 
encounters with an injection prescribed, percentage of medicines prescribed from essential medicines list); 

2. patient care indicators (average consultation time, average dispensing time, percentage of medicines actually 
dispensed, percentage of medicines adequately labelled, the patient’s knowledge of correct dosage); 

3. facility indicators (availability of a copy of essential medicines list of formulary, availability of key medicines). 

A recently conducted quantitative review of studies published between 1990 and 2007, (that reported common 
indicators of medicines use) reported on medicines use in developing and transitional countries, and on the 
impact of interventions undertaken to improve medicines use, by analysing data from 679 studies conducted 
in 97 countries (10). 

To accompany the Questionnaire on structures and processes of country pharmaceutical situations, WHO has 
developed a set of facility-level indicators to measure key outcomes of these structures and processes in the 
areas of access, product quality and rational use (11, 12). 

• Access is measured in terms of the availability and affordability of essential medicines. 

• Quality is represented by the absence of expired stock on pharmacy shelves and adequate handling and 
conservation conditions. 

• Rational use is measured by examining prescribing and dispensing practices and the implementation of 
strategies that have been shown to support rational use, such as standard treatment guidelines and the 
essential medicines list. 

These indicators are measured with standardized collection instruments in public health facilities, private 
drug outlets and in warehouses supplying the public sector, through the Survey of Medicine Prices, Availability, 
Affordability and Price Components (6). Surveys of 30 public health facilities and their dispensaries gathered 
information about the availability of essential medicines, medicine prices, adequacy of conservation conditions, 
affordability, prescribing and dispensing habits, and presence of guidelines. A similar survey of five warehouses 
supplying the public sector also examined availability, stockout duration, and adequacy of conservation 
conditions. Surveys of 30 private pharmaceutical outlets assessed the availability, affordability and prices of 
medicines. 
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Key informant surveys 

Surveys by experts with extensive knowledge about the medicines situation in a country can be used to generate 
information about pharmaceutical policies and practices related to regulation, selection of essential medicines, 
as well as procurement and use. While this method has a low cost and is relatively easy to implement, the 
disadvantage is its subjectivity, which introduces measurement errors and affects comparability both between 
countries and over time within the same country 

Data on national medicines policies and their components (including legislation and regulations, quality 
control of medicines, essential medicines lists, supply systems, financing, access to medicines, 
production, rational use, and protection of intellectual property rights) can be obtained from the WHO 
questionnaire on structures and processes of country pharmaceutical situations (8, 9). The questionnaire is a basic 
assessment tool that provides a rapid means of obtaining information on the existing infrastructure and key 
processes of each component of the pharmaceutical sector. Data are collected through a country data collection 
instrument that are used to generate a Pharmaceutical Country Profile . The national coordinator identifies 
responsible people, government agencies or groups who can provide responses and source documents 
related to the different pharmaceutical sections/areas in the data collection instrument. 

4.3 Core indicators 

The recommended core indicators to measure access to essential medicines are as follows. 

Recommended core indicator 1: Average availability of 74 selected essential medicines 
in public and private health facilities 

Definition 

The average percentage of medicines outlets, where a selection of essential medicines are found on the day of 
the survey. 

Data collection methodology 

National surveys 3 of medicine price and availability conducted using a standard methodology developed by 
WHO and Health Action International. 4 Data on the availability of a specific list of medicines are collected 
from six geographic or administrative areas in a sample of medicine dispensing points. 

Periodicity of measurement 

In the absence of routine monitoring, it is recommended that a national survey of medicine prices and availability 
be conducted every three to five years using the WHO/HAI standard methodology. 


3 In large countries such as India and China, sub-national surveys have been conducted. 

4 http://www.haiweb.org/medicineprices/manual/documents.html, accessed May 23,2010. 
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Recommended core indicator 2: Median consumer price ratio of 14 selected essential 
medicines in public and private health facilities 

Definition 

Consumer price ratios are calculated as the ratio between median unit prices (e.g. price per tablet or therapeutic 
unit) and Management Sciences for Health (MSH) median international reference prices 5 for that exact product 
for the year preceding the survey. 

Data collection methodology 

National surveys of medicine price and availability conducted using a standard methodology developed by 
WHO and HAI. Data on the price of a specific list of medicines are collected in six geographic or administrative 
areas in a sample of medicine dispensing points. 

Periodicity of measurement 

In the absence of routine monitoring, it is recommended that a national survey of medicine prices and availability 
be conducted every three to five years using the WHO/HAI standard methodology. 

4.4 Additional indicators for a full pharmaceutical profile 

In addition to the core indicators, the following indicators are recommended should a country wish to undertake 
a full pharmaceutical profile. These are in line with the monitoring of progress towards MDG indicator 8.13 
— the proportion of population with access to affordable essential medicines on a sustainable basis — that 
has been reported by the United Nations in 2008 (4) and 2009 (5) using a set of nine structural and process 
indicators proposed by WHO to quantify access (see summary in Table 4.2). 

Recommended indicator 7 ; Access to essential medicines/technologies as part of the 
fulfillment of the right to health, recognized in the constitution or national legislation 

Definition 

Whether or not access to essential medicines/technologies is recognized in the constitution or national 
legislation as part of the progressive realization of the right to health and/or as a specific entitlement of all 
citizens. 

Data collection methodology 

This indicator necessitates a review of the national constitution or legislation. At the international level, such a 
review was conducted by WHO in 2008. 

Periodicity of measurement 

The data are not likely to change frequently over time. Four-yearly national updates are envisaged as part of 
WHO’s global pharmaceutical surveys. 


5 MSH international reference prices have been selected as a comparator as they are widely available, updated frequently, and relatively 
stable over time. They represent median prices of high quality multi-source medicines offered to developing and middle-income 
countries by different suppliers. The large majority of MSH prices are for multi-source products, and are usually ‘Ex-Works’ prices. 
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Recommended indicator 2: Existence and year of last update of a published national 
medicines policy 

Definition 

Existence of an official National Medicines Policy (NMP), expressed as a yes/no indicator, and updated within 
the past five years. 

Data collection methodology 

Key informant surveys with a standard tool, such as the WHO Questionnaire on structures and processes of 
country pharmaceutical situations. 

Periodicity of measurement 

Every four years. 

Recommended indicator 3: Existence and year of last update of a published national list 
of essential medicines 

Definition 

Existence of an essential medicines list expressed as a yes/no indicator and updated within the past five years. 

Data collection methodology 

Key informant surveys with a standard tool, such as the WHO Questionnaire on structures and processes of 
country pharmaceutical situations. 

Periodicity of measurement 

Every four years. 

Recommended indicator 4: Legal provisions to allow/encourage generic substitution in 
the private sector 

Definition 

Existence of legal provisions to allow generic substitution in the private sector, expressed as a yes/no indicator. 

Data collection methodology 

Key informant surveys with a standard tool, such as the WHO questionnaire on structures and processes of 
country pharmaceutical situations. 

Periodicity of measurement 

Every four years. 
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Recommended indicator 5: Public and private per capita expenditure on medicines 

Definition 

The reference indicator is total pharmaceutical expenditure (TPE). It may be defined as the total consumption 
of pharmaceuticals, regardless of the distribution mean, the place or condition of consumption or its type 
(prescription or over-the-counter). Per capita data are obtained from the whole population. As much as 
possible, this indicator is disaggregated into two components to reflect public and private sector financing. 
Public financing refers to social security, territorial governments, and extrabudgetary entities combined, while 
private financing includes out-of-pocket spending, finances related to private insurance, nongovernmental 
organizations, and corporations (excluding social security). 

Data collection methodology 

Data on medicines expenditures can be obtained from National Health Accounts (NHA), 6 which is a systematic, 
comprehensive, and consistent monitoring of resource flows in a country’s health system for a given period. 
The NHA is designed to capture the lull range of information contained in resource flows and reflects the 
main functions of health care financing, such as resource mobilization and allocation, pooling and insurance, 
purchasing of care, and the distribution of benefits. More information on NHA can be found in Section 5 of 
this toolkit, health systems financing. While the NHA has different tables with the same content from various 
approaches, TPE should be the basis of the estimation for this indicator. 

Periodicity of measurement 

Periodicity is dependant on collection of NHA data; annually for most countries. 

Recommended indicator 6: Percentage of population covered by health insurance 

Definition 

Numerator: Number of people covered by health insurance. 

Denominator: Total number of population. 

Data collection methodology 

Household survey. For example, data on insurance coverage has been reported using data from the World 
Health Survey 2004. 7 

Recommended core indicator 7: Percentage mark-up between manufacturers' and 
consumer prices 

Definition 

How much the final medicine price is greater, in percentage, above the manufacturer’s selling price or the cost, 
insurance and freight price. 


6 http://www.who.int/nha/en/, accessed May 23,2010. 

7 http://www.who.int/healthinfo/survey/en/index.html, accessed May 23, 2010. 
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Data collection methodology 

National surveys of medicines prices and availability conducted using a standard methodology developed by 
WHO and HAI. Data on the add-on costs that contribute to the final price of medicines are collected by tracking 
selected tracer medicines through the supply and distribution chain. 


Periodicity of measurement 

It is recommended that a national survey of medicines prices and availability be conducted every three to five 
years using the WHO/HAI standard methodology. 


Table 4.2 Summary of indicators for a full pharmaceutical profile, including core indicators for 
access to essential medicines 


Indicators 

Data collection method 

Structure 

1. Access to essential medicines/technologies as part of the fulfilment of the right to 
health, recognized in the constitution or national legislation. 

Review of national constitution or legislation. 

2. Existence and year of last update of a published national medicines policy. 

3. Existence and year of last update of a published national list of essential medicines 

4. Legal provisions to allow/encourage generic substitution in the private sector 

Key-informant surveys using standard tool such as the 

WHO Questionnaire or structures and processes of country 
pharmaceutical situations. 

Process 

5. Public and private per capita expenditure on medicines 

National Health Accounts 

6. Percentage of population covered by health insurance 

Household surveys 

7. Average availability of 14 selected essential medicines in public and private health 
facilities* 

8. Median consumer price ratio of 14 selected essential medicines in public and private 
health facilities* 

National (or sub-national when necessary) surveys of 
medicine price and availability conducted using a standard 
methodology developed by WHO and Health Action 
International. 

9. Percentage mark-up between manufacturers'and consumer prices 



* Core indicators to measure access to essential medicines. 


Selected tools 

Measuring medicine prices, availability, affordability and price components, 2nd ed. Geneva, World Health 
Organization and Health Action International, 2008 (http://apps.who.int/medicinedocs/index/assoc/sl4868e/ 
sl4868e.pdf, accessed on 4 April 2010). 

This second edition includes updated versions of the survey manual, an automated data workbook, survey 
instruments and a CD ROM of survey tools and background materials, all of which have been refined 
based on the lessons learnt in the 50+ surveys conducted to date. 

WHO operational package for monitoring and assessing country pharmaceutical situations. Guide for coordinators 
and data collectors. Geneva, World Health Organization, 2007. (http://www.who.int/medicines/publications/ 
WHO_TCM_2007.2/en/index.html, accessed on 4 April 2010). 

This operational package is a tool for researchers, policy-makers, planners and others who need to use 
standardized measurement tools to gather data and other information for monitoring and assessing country 
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pharmaceutical situations. Level I questionnaire can be used as a checklist to illustrate sectoral structures, 
strategies and approaches. Countries can also use selected forms from the Level II facility survey in their 
routine monitoring. The results can help focus their strategies, advocacy plans and information campaigns. 

Note This package is being updated and a new version will be published in early 2011 

How to investigate drug use in health facilities. Geneva, World Health Organization, 1993 (http://apps.who.int/ 
medicinedocs/en/d/Js2289e/, accessed on 4 April 2010). 

This manual defines a limited number of objective measures that can describe the medicine use situation 
in a country, region or individual health facility. The medicine use indicators described are intended to 
measure specific aspects of the behaviour of health providers in health facilities in a reproducible manner. 

Logistics indicators assessment tool (LIAT). Arlington, VA, USAID DELIVER Project, 2008 (http://deliver.jsi. 
com/dhome/topics/monitoring/monitoringpubs/meresources/metools, accessed on 4 April 2010). 

This quantitative data collection instrument, developed by the USAID DELIVER project, helps assess 
health commodity logistics system performance and commodity availability at health facilities. A detailed 
user guide is included. 

Logistics system assessment tool (LSAT). Arlington, VA, John Snow Inc./DELIVER for the United States 
Agency for International Development, revised ed., 2008 (http://deliver.jsi.com/dhome/topics/monitoring/ 
monitoringpubs/meresources/metools, accessed on 4 April 2010). 

This qualitative data collection instrument provides a comprehensive system-level assessment of logistics 
system performance for any programme that manages a health commodity. 

Further reading 

WHO model list of essential medicines. 16 th List, March 2009-Unedited version, (http://www.who.int/selection_ 
medicines/committees/expert/17/WEB_unedited_16th_LIST.pdf, accessed on 4 April 2010). 
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Annex. Recommended list of medicines for inclusion in the WHO service 
availability and readiness assessment methodology 


Core medicines to be included in all surveys 


Amitriptyline 25 mg capsule/tablet 

Diazepam 5 mg capsule/tablet 

Amoxicillin 500 mg capsule/tablet 

Diclofenac 50 mg capsule/tablet 

Atenolol 50 mg capsule/tablet 

Glibenclamide 5 mg capsule/tablet 

Captopril 25 mg capsule/tablet 

Omeprazole 20 mg capsule/tablet 

Ceftriaxone 1 g/vial injection 

Paracetamol 24 mg/ml suspension 

Ciprofloxacin 500 mg capsule/tablet 

Salbutamol 0.1 mg/dose inhaler 

Co-trimoxazole 8+40 mg/ml suspension 

Simvastatin 20 mg capsule/tablet 

Additional medicines to be considered for inclusion 

Infectious diseases 

Tuberculosis 

1. Co-trimoxazole (capsule/tablet) 

1. Ethambutol 

2. Fluconazole 

2. Isoniazid 

3. Albendazole or Mebendazole (depending on country standards) 

3. Pyrazinamide 

4. Metronidazole 

4. Rifampicin 

Chronic diseases 

5. Streptomycin (injectable) 

1. Enalapril (depending on country standards) 

6. Isoniazid + Rifampicin (2FDC) 

2. Bedometasone (inhaler) 

7. Isoniazid + Ethambutol (EH) (2FDC) 

3. Metformin 

8. Isoniazid + Rifamipicin + Pyrazimamide (RHZ) (3FDC) 

4. Insulin (injection) 

9. Isoniazid + Rifamipicin + Pyrazimamide + Ethambutol (4FDC) 

Other 

Antiretrovirals 

1. Ibuprofen 

1. Zidovudine (AZT, ZDV) 

2. Oral rehydration salts (sachets) 

2. Abacavir (ABC) 

3. Measles vaccine 

3. Didanosine (DDI) 


Reproductive health 

1. Oral contraceptive pills (combined) 

2. Injectable contraceptives (progestin-only) 

3. Condoms (male) 

4. Oxytocin (injection) 

5. Magnesium sulphate (injection, eclampsia) 

6. Diazepam (injection) 

Malaria 

1. ACT, Artemeter + Lumefantrine 

2. SP, Sulphadoxine + Pyrimethamine 

3. Quinine (oral or injectable) 

4. Other antimalarial medicines (oral or injectable) 


4. Eifravenz (EFZ) 

5. Lamivudine (3TC) 

6. Nevirapine (NVP) 

7. Stavudine 40 or 30(d4T) 

8. D4T + 3TC 

9. D4T + 3TC + NVP 

10. AZT + 3TC 

11. AZT + 3TC +ABC 

12. AZT + 3TC + NVP 

13. Tenofovir + Disoproxil fumarate (TDF/Viread) 

14. TDF + Emtricitabine (FTC) 

15. TDF + 3TC 

16. TDF + 3TC + EFV 

17. TDF + FTC + EFV 


Protease inhibitors 

1. Atazanavir (ATV) 

2. Indinavir (IDV) 

3. Lopinavir/Ritonavir (LPV/RTV) 

4. Nelfinavir(NFV) 

5. Ritonvir (RTV) 

6. Saguinavir (SQV) 


Source: Service availability and readiness assessment methodology, upcoming, http://www.who.int/healthinfo/systems/serviceavailabilitymapping/en/ 
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5. Health systems financing 














Health systems financing 

5.1 Introduction 

Health financing is fundamental to the ability of health systems to maintain and improve human welfare. At the 
extreme, without the necessary funds no health workers would be employed, no medicines would be available 
and no health promotion or prevention would take place. However, financing is much more than a simple 
generation of funds (see Box 5.1). To understand the nature of indicators that can be used to monitor and 
evaluate health systems financing requires explicit assessment of what it is expected to achieve. 



Box 5.1 What is health financing? 


Health financing refers to the “function of a health system concerned with the mobilization, 
accumulation and allocation of money to cover the health needs of the people, individually and 
collectively, in the health system... the purpose of health financing is to make funding available, as 
well as to set the right financial incentives to providers, to ensure that all individuals have access 
to effective public health and personal health care” (1). 


While the goals of health systems financing can be expressed in various ways, there is a general consensus 
that it should not only seek to raise sufficient funds for health, but should do so in a way that allows people 
to use the needed services without the risk of severe financial hardship (often called financial catastrophe or 
impoverishment). 1 This involves the accomplishment of two related objectives: (i) to raise sufficient funds and 
(ii) to provide financial risk protection to the population. These objectives can be achieved more easily if the 
available funds are used efficiently, highlighting the need for a third objective, that of efficiency in resource 
utilization. As a result, the financing system is often divided conceptually into three inter-related functions 
— (i) revenue collection, (ii) fund pooling, and (iii) purchasing/provision of services. Before focusing on 
measurement strategies and indicators for these functions it is important to understand their key components. 

In most low-income and many middle-income countries, revenue collection derives from a mix of domestic 
and external sources. Despite the substantial increases in external assistance for health since 2000, the available 
resources are still insufficient in most low-income settings to assure universal coverage with even a very basic 
set of needed interventions. The adjustment of Commission on Macroeconomics and Health estimates of the 
cost of a core package to current prices reveals a need for around US$ 40 per person per year. This is an 


1 In 2005, WHO Member States endorsed a resolution urging governments to develop health financing systems aimed at attaining 
and maintaining “universal coverage” — described as raising sufficient funds for health in a way that allows access to needed services 
without the risk of a financial catastrophe. 
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underestimate for many reasons, 2 but even then, almost a third of the 193 member countries of WHO did not 
have access to even this level of funding in 2005, and 33 spend less than US$ 25 per person per year despite 
increased external inflows. An ideal indicator for revenue collection would need to capture the amount and 
adequacy of the funds that are raised. 

Financial risk protection is determined by how funds are raised, and whether and how they are pooled to spread 
the risk across population groups. Direct user-charges, for example, are regressive, i.e. the rich pay the same fees 
as the poor, which deters some people from seeking or continuing care. The funds also do not provide financial 
risk protection, in that people pay when they are sick and do not pay when they are healthy. As a result of this 
lack of solidarity, some people incur financial hardships and may even be pushed below the poverty line. A 
financing policy must grapple with the question of how to raise funds equitably, which usually implies a degree 
of progressivity (where the rich contribute a higher proportion of their income than the poor). It also needs 
to consider how to ensure access to needed services while protecting people against the more severe financial 
consequences of paying for care. These goals cannot be achieved without some form of prepayment and the 
subsequent pooling of the collected revenues, i.e. people pay into a pool when they are healthy and can draw 
on these funds when they are sick. Pooled funds can be derived from tax or health insurance contributions and 
in most countries they come from a mix of sources. Indicators in this area need to capture the extent to which 
people are protected from the financial risks associated with ill health. It would also be valuable to measure the 
extent of progressivity in the way that prepaid funds for health (e.g. taxes and insurance premiums) are raised. 

Ensuring efficiency in resource use is a complex issue that should address questions on how to reduce waste 
and corruption; what interventions should be available for the existing resources; whether services should 
be provided by the government or purchased from the non-government sector; how providers (e.g. health 
workers, hospitals, etc.) should be paid to ensure quality and efficiency; and whether specific types of services 
or incentives should be targeted at the poor. Thus, because of the multiple dimensions, it is not particularly easy 
to define a single, easily understandable indicator of efficiency for health system financing. 

5.2 Sources of information on health systems financing 

A national government’s total budget and the part allocated to health are both usually public information, and 
can be used to evaluate the government’s total commitment to health as well as in proportion to other priorities. 
A planned budget however, while an important indicator of commitment, can differ significantly from the 
funds that are eventually released to departments and the subsequent expenditures. 

In most countries, information on government health expenditures channelled through the ministry of health 
is usually available through the ministry of finance or regional authorities in decentralized systems. However, 
information on government health expenditures that are channelled through non-health ministries, such as 
military or police health services, are sometimes more difficult to obtain. While budget information is available 
in “real time”, there is often a delay of perhaps about a year in the production of consolidated expenditure 
accounts. Public expenditure reviews, if available, are often an excellent source of information. These reviews 
collate information from various sources that help to determine whether government expenditures follow the 
budget plans and stated strategic objectives. Sometimes these reviews seek to examine the efficiency of resource 


2 The original estimates did not include antiretroviral drugs for HIV, interventions for non-communicable diseases or a variety of 
health system strengthening costs essential to being able to deliver the package. Moreover, it assumes that only the interventions in 
the core set will be provided. 
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use, though in very broad terms, as well as the ability of the financial management and accounting systems and 
institutions to track expenditures. 3 

Information on commitments to official development assistance for health made by donor countries, 
international organizations and some foundations have been collated by the OECD for many years and 
reported since 2002. 4 However, this information, which is available by donor and by recipient country, should 
be used cautiously Firstly, a part of the reported disbursements (and a large part in some cases) does not reach 
the recipient countries and should not be included in estimates of country health expenditure. These include 
payments for technical support to countries, payments generally made to nationals of countries other than 
the recipient country, and funds that are generally spent outside the recipient country. Secondly, there is an 
increasing move towards general budget support to countries, which is difficult to allocate to the different 
sectors. General budget support is reported in a separate section in the OECD database, and a method of 
allocation between the different sectors needs to be devised. Thirdly, emerging donors such as China and India, 
and some private philanthropists, are not included in the database. 

It is better to track expenditure from external sources at the country level, but this is often difficult especially 
where this funding is channelled through non-governmental organizations (NGOs) or the private sector. 
Many countries do not require external donors or NGOs to report their in-country expenditures, or if they are 
required to submit budgets with proposals at the time they gain permission to work in the country, there is no 
database where this information is systematically captured nor where actual expenditures are recorded. This 
also applies to domestic NGOs and other charitable organizations supporting the health sector, where it is often 
difficult to track expenditures. 

National-level expenditures as a result of third-party payments (e.g. from insurance and/or social security) may 
be available from fund managers. If third-party payers are primarily small community-based organizations, 
such as community-based health insurance funds, then compiling expenditure information is much more 
difficult. 

Information on household out-of-pocket expenditures is only available from household surveys. The World 
Bank has sponsored Living Standards Measurement Surveys since 1980 from which information on household 
health expenditures can be extracted, 5 and World Health Surveys sponsored by WHO in 2000-2001 6 also 
contain a household expenditure module. 7 Many countries undertake household income and/or expenditure 
surveys from which information on health expenditures can be gleaned. There is considerable variability in 
the types of questions used to obtain household health expenditures, which makes comparisons over time 
and across countries difficult. As a long-term goal it is important to choose a standard instrument that would 
enhance comparability, either for independent surveys or to piggyback onto other household surveys carried 
out for other reasons. 

National Health Accounts 

The best source of health expenditure data is from NHA, which combines expenditure data from all sources 
and through all types of financial agents. The System of Health Accounts (SHA) developed by the OECD for 

3 Probably the bulk of public expenditure reviews have been sponsored by the World Bank and UK Department For International 
Development to date — see, for example, http://www.opml.co.uk/services/public_expenditure_reviews/index.html. 

4 http://stats.oecd.org/wbos/Default.aspx?usercontext=sourceoecd 

5 http://www.worldbank.org/LSMS 

6 http://www.who.int/healthinfo/survey/whsresults/en/index.html 

7 Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) include modules on household assets, but 
not expenditures. 


Health systems financing 



its countries has become the international classification standard although some country analysts prefer to use 
variations on this theme, including an approach called ‘national account sub-accounts’. In general, it is possible 
to modify the figures emerging from one method to make them consistent with the other. Recently, the WHO, 
World Bank and USAID jointly developed a guide to undertake NHA in low-income countries based on SHA, 
adapted to meet the needs of low-income countries. 8 Application of the methods in a variety of settings has 
resulted in a group effort between the OECD, Eurostat and WHO to revise the SHA with the goal of making it 
more appropriate to countries at all income levels. 9 

National Health Accounts studies vary among countries — some countries have undertaken regular NHA 
studies, some have undertaken one or two studies but not regularly, while still others have yet to undertake 
a full NHA exercise. In the last case, data on health expenditures need to be collated from various sources. 
WHO works with countries to collate information from these sources, which combined with the information 
provided by countries who have undertaken NHA studies, allows annual reports of selected health expenditure 
aggregates for 192 of its 193 member countries. 10 These figures also form the basis of the health expenditure 
data reported in the World Bank’s World Development Indicators. 11 

Various organizations provide support to countries seeking to develop better information on health expenditures, 
such as the USAID’s Health Systems 20/20 project, WHO and the Swedish International Development 
Cooperation Agency. However, full NHA analyses have yet to be institutionalized in all countries. 

5.3 Core indicators 

Building on the discussion in Section 1, the core indicators for the availability of funds and the extent of financial 
risk protection have been agreed on at various fora. 12 

Recommended core indicator la: Total expenditure on health 

This indicator provides information on the overall availability of funds. Sufficiency must be considered as a 
second step, in relation to country-specific estimates of the funds needed to ensure access to the desired level of 
services, or in terms of comparisons with other countries with similar levels of gross domestic product (GDP) 
per head. Some countries also seek to compare their total expenditure on health as a proportion of GDP with 
those in other countries. This is included in Table 5.1 as a possible additional indicator. 

Definition 

• Numerator: The sum of all health expenditures (ideally from NHA and including all sources of 
funds — external, government, and non-government including household out-of-pocket payments). 

• Denominator: Total population. 

Data collection methodology 

Data collection is through country-specific reporting by the ministry of finance/ministry of health/other 
relevant ministries (for government expenditures), donors (for funding that is not channelled through the 


8 www.who.int/nha/docs/English_PG.pdf 

9 www.who.int/nha/methods/en/index.html 

10 www.who.int/nha 

11 http://go.worldbank.org/U0FSM7AQ40 

12 Health System Metrics Technical meeting, September 28-29, 2006, Glion. 


Health systems financing © 



ministry of finance/ministry of health), insurance fund managers (for third-party funding) and household 
surveys (for out-of-pocket expenditures) using NHA methodology. Population numbers should ideally be de 
facto rather than de jure population, with the most complete cross country source being the United Nations 
Population Division. 

Periodicity 

Health expenditures should ideally be calculated on an annual basis. Full surveys of household expenditure are 
expensive and could be done less frequently, with extrapolations in the inter-survey years. 

Cost 

The cost of initially producing NHA varies considerably depending on the information and bureaucratic 
structure already available and the need for external technical assistance. Experience in some countries has 
shown that the costs to pull together existing information for the first NHA could be as low as US$ 50 000-75 000 
with subsequent yearly costs largely related to producing recurrent statistics. This assumes that household 
expenditure surveys are already available and that international consultants do not do the bulk of the work. 
Initial costs include: a) training personnel; b) ensuring adequate computers and office infrastructure; c) logistics 
related to explanatory meetings and training on completing reporting forms or collecting information; and d) 
development of report templates relevant for national planning (2). 

Recommended core indicator lb: General government expenditure on health as a 
proportion of general government expenditure (GGHE/GGE) 

This indicator is related to how much funding is raised for health and reflects government commitment. African 
heads of state committed to ensuring that 15% of overall government expenditure goes to health in the Abuja 
Declaration of 2001 (3). This can be taken as an aspirational goal, which even a few of the richer countries in 
the world have yet to achieve. While it is difficult to justify why 15% is the ideal cut point, many countries still 
devote less than 4% of GGE to health, suggesting low levels of government commitment. 

Recommended core indicator 2: The ratio of household out-of-pocket payments for 
health to total expenditure on health 

The ideal indicator of financial risk protection is the proportion of the population incurring catastrophic health 
expenditure due to out-of-pocket payments. A variation is the percentage that is impoverished as a result of 
out-of-pocket payments. 

WHO has defined financial catastrophe for the past eight years as direct out-of-pocket payment exceeding 
40% of household income net of subsistence needs. Subsistence needs are taken to be the median of household 
food expenditure in the country. Expenditures in excess of the 40% cut point generally require reallocation of 
household expenditures from basic needs, sometimes even from childrens education (4). The World Bank now 
has a simpler definition of financial catastrophe, i.e. occurring when out-of-pocket payment exceeds 10% of a 
households total income. While this does not incorporate the progressivity allowed by the deduction of basic 
subsistence needs, it is probably simpler to estimate and similar to those derived by the WHO method. 

To explore questions of equity, it may be possible in most cases to estimate the incidence of financial catastrophe 
by income quintile, or by wealth quintile if a separate wealth or asset index can be constructed from the same 
household survey. Indeed, in most developing countries, self-reported total expenditure is regarded as a more 
reliable indicator of command over resources than self-reported income, and thus these comparisons are usually 
made in terms of total expenditure quintiles (5). Such comparisons however need to be interpreted carefully. In 
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many countries the quintile with the lowest income (or lowest level of total expenditure) has a lower incidence 
of catastrophic payments than richer quintiles. This reflects the perverse nature of user fees. When people are 
very poor, they do not use services for which they have to pay, and thus do not suffer a financial catastrophe. 
As they become slightly richer, they begin to use services, but then suffer the adverse financial consequences 
linked to paying for care. 

Definition 

The number of households in each region where direct out-of-pocket payments to providers for health 
during the past 12 months was more than 40% of their household income net of subsistence, or 10% of their 
total income. 

• Numerator: Household out-of-pocket expenditure for health during the past 12 months. 

• Denominator: Household income. As argued above, in most developing countries it is accepted that self- 
reported total expenditure on health is a more reliable indicator of household purchasing power than 
self-reported income, so this should be used as the denominator in those settings. 

Data collection methodology 

Household interview surveys. 

Periodicity 

The ratio is not likely to change dramatically over time unless substantial health financing reforms are done. In 
most countries, measurements done every five years would be adequate. 

Cost 

The cost for undertaking a national level household survey with a sample size sufficient for regional level 
disaggregation specifically for the purpose of collecting health expenditure data varies widely depending on the 
existing in-country capacity. The cost range maybe from US$ 350 000 to US$ 1 000 000 depending on the level 
of technical support required. However, usually health expenditure data would be collected as part of a broader 
income and expenditure survey, or as an added module in a broader health survey. Accordingly, the additional 
costs are likely to be relatively small. The main new cost will be incurred by personnel who analyse the data and 
produce the information for policy makers. 

Despite the logic of using the incidence of financial catastrophe as the core indicator, it is argued that a simpler 
indicator of financial risk protection is the ratio of out-of-pocket spending to total expenditure on health (OOP/ 
THE) — or the inverse, the ratio of prepaid expenditures (taxes and insurance) to total expenditure on health. 
Undoubtedly, there is a high correlation between this indicator and the incidence of financial catastrophe (and 
impoverishment), and therefore is included as the core indicator here. 

While the indicator may appear simpler, it requires exactly the same data from household expenditure surveys 
as the indicator on financial catastrophe described above. So if the surveys to estimate OOP/THE are available, 
they can also estimate the incidence of financial catastrophe. Experience has shown that policy makers can 
immediately perceive the political relevance of the incidence of financial catastrophe and/or impoverishment, 
whereas OOP/THE may not have the same immediate policy impact. For the purposes of discussion, at this 
stage OOP/THE is used as the recommended indicator in Table 5.1, with the incidence of financial catastrophe 
as an optional indicator. However, the ordering preference can easily be reversed. 
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Indicator to capture efficiency of the health financing system 

At this stage, a core indicator to capture the efficiency of the health financing system is not being recommended 
because it is difficult to define a single indicator that is relatively simple to measure and easy to interpret. 
The proportion of total government health expenditure spent on salaries is included as one possible optional 
indicator, but needs to be interpreted very carefully Certainly if this proportion is very high, health workers 
would not have sufficient drugs or other inputs to be able to do their jobs properly However in some countries, 
where governments choose to contract out the provision of services to the private sector or NGOs rather than 
employ their own personnel, the proportion spent on salaries is low because payments to external contractors 
do not appear as salaries. Thus, this is not a very useful indicator of efficiency. 

Certain optional indicators that could be measured depending on the capacity of the country are summarized 
in Table 5.1. Some of these reflect processes or outputs, while others are more related to outcomes. 

5.4 Needs assessment for institutionalizing collection of data for monitoring 
finance indicators 

Since total expenditure on health is currently being reported for 192 of the 193 WHO member countries, the 
primary need is to improve the quality of information that is already being collected, and to strengthen the 
institutionalization of the generation and utilization of this information. This requires regular and accurate 
reporting of government expenditures at all levels of the government, regular household expenditure surveys, 
and some method of routinely tracking expenditures by NGOs, faith-based organizations, philanthropies and 
the private sector. 

WHO has identified four steps essential to the institutionalization process of NHA (2). These are: (i) creating a 
demand on the part of policymakers for institutionalization; (ii) determining a location where NHA is housed; 
(iii) establishing standards for data collection and analysis; and (iv) instituting data reporting requirements. 

The institutionalizing process of NHA also requires an assessment of existing infrastructure and systems and 
should include the following critical information. 

• Government and stakeholder commitment to NHA as indicated by such steps as delegation of 
responsibility for generating NHA to a specified body and allocation of a budget for implementation. 

• An assessment of existing human resources numbers and capacity, and infrastructure for generating NHA 
data. 

• Clarity of health financing mechanisms related to funding sources, processes for channelling funds, and 
knowledge on where information on external health funding and third-party funding is available including 
if it is provided to any central or coordinating body. An assessment of the process currently used by WHO 
for NHA estimates for the country and identification of which data are weakest or least reliable should help 
obtain this information. 

• Identification of problems with regards to transparency in national or donor health funding, and the need 
for policy changes or advocacy to improve this. 

• Development of an audit function within the NHA to periodically assess the completeness and accuracy 
of the submitted or collected information, with a systematic strategy for feedback to the data sources to 
improve availability and quality of needed information. 
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5.5 Using financial indicators for health systems strengthening 

In general, total expenditure on health should be increasing both in absolute terms and as a proportion of GDP 
in low-income countries, while the proportion of households facing financial catastrophe as a result of out-of- 
pocket payments should be decreasing. Financial indicators could be used to answer the questions listed below. 

Is the total expenditure on health per capita, within the range defined internationally, enough to allow universal 
coverage of key health interventions (e.g. at least US$ 40 per capita)? 

Is the percentage of the national budget for health reasonable given the national situation? Does it reflect a 
strong government commitment to health? 

What proportion of total expenditure on health is dependent on external funding, and may not be sustainable 
in the long run? What steps can be taken domestically to raise additional funds for health? 

Does a high total expenditure on health get reflected in health outcomes? If not, the efficiency and quality of 
service, and possibly transparency and corruption issues need to be reviewed. 

2. What policies or implementation practices are needed to decrease catastrophic expenditures? 

What does the assessment of out-of-pocket catastrophic expenditure show in terms of health finance 
mechanisms that contribute to, or hurt, equity in financing health? What other options are available to 
improve equity? 

Are the existing health finance policies being implemented in a transparent manner (e.g. are the households 
receiving exemptions or subsidized services and medicines if they are eligible?). 

Are there regional disparities that need to be addressed separately? 
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Table 5.1 Recommended indicators on health systems financing 
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Selected tools 

National Health Accounts (NHA). Geneva, World Health Organization (http://www.who.int/nha, accessed 
February 10, 2010). 

Health financing policy requires decisions on how to raise funds, how to pool them, and how to use 
them equitably and efficiently. Informed decision-making requires reliable information on the quantity 
of financial resources used for health, their sources and the way they are used. National Health Accounts 
provides evidence to monitor trends in health spending for all - public and private sectors, different 
health-care activities, providers, diseases, population groups and regions in a country. It helps in 
developing national strategies for effective health financing and in raising additional funds for health. 
The information can be used to make financial projections of a country’s health system requirements and 
compare its experiences with those in the past or with those of other countries. 

Guide to producing national health accounts - with special applications for low-income and middle-income 
countries. Geneva, World Health Organization, World Bank, USAID, 2005 (http://www.who.int/nha/docs/ 
English_PG.pdf, accessed March 29, 2010). 

This Guide walks the reader through the process of acquiring and evaluating data for National Health 
Accounts, and provides step-by-step examples on translating numbers into information useful for policy 
analysis and development. 

OASIS (forthcoming) (http://www.who.int/health_financing/tools) 

The Organizational Assessment for Improving & Strengthening Health Financing can be used to: (i) 
analyse the performance of a health financing system by assessing key design issues and implementation, 
(ii) identify bottlenecks in the functioning of institutions and organizations and (iii) help in finding 
institutional and organizational alternatives. 

Further reading 

Health expenditure 

A system of health accounts. Paris, Organisation for Economic Co-operation and Development, 2000 (http:// 
www.oecd.org/health/sha, accessed February 10, 2010). 

Financing policy 

Kutzin J. A descriptive frameworkfor country-level analysis of health care financing arrangements. Health Policy, 
2001, 56:171-204. (http://www.journals.elsevierhealth.eom/periodicals/heap/artide/PIIS0168851000001494, 
accessed February 10, 2010). 

Approaching health financing policy in the European Region. World Health Organization, Regional Committee 
for Europe, Fifty-sixth session, Copenhagen, 2006 (EUR/RC56/BD/1; http://www.euro.who.int/Governance/ 
RC/RC56/20060622_1, accessed February 10, 2010). 

Kirigia JM, et al. An overview of health financing patterns and the way forward in the WHO African Region. 
The East African Medical Journal, 83, 2006 (Supplement) (http://www.who.int/health_financing/documents/ 
list/en/index2.html, accessed February 10, 2010). 
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Carrin G, Waelkens MP, Criel B. Community-based health insurance in developing countries: a study 
of its contribution to the performance of health financing systems. Tropical Medicine and International 
Health, 2005, 10:799-811. (http://www3.interscience.wiley.com/journal/118667685/issue, accessed 

February 10, 2010). 

Contracting and health services - special theme issue. Bulletin of the World Health Organization, 2006, 84:841- 
920. (http://www.who.int/bulletin/volumes/84/ll, accessed February 10, 2010). 

Health financing: a strategy for the African region. World Health Organization, Regional Committee for Africa, 
Fifty-sixth session, Addis Ababa, 2006 (AFR/RC56/10; http://www.who.int/health_financing/documents/cov- 
afrostrategy, accessed March 27, 2010). 

Kutzin J. Health financing policy: a guide for decision-makers. Health Financing Policy Paper. Copenhagen, 
WHO Regional Office for Europe, 2008 (http://www.euro.who.int/document/e91422.pdf; accessed February 
10 , 2010 ). 

Gottret P, Schieber G. Health financing revisited: a practitioner’s guide. World Bank, 2006 (http://web. 
worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTHEALTHNUTRITIONANDPOPULATION/EXTH 
SD/0„contentMDK:20200211~menuPK:414634~pagePK:210058~piPK:210062~theSitePK:376793,00.html, 
accessed February 10, 2010). 

Hutton G, Baltussen R. Cost valuation in resource-poor settings. Health Policy and Planning, 2005, 20:252-259 
(http://heapol.oxfordjournals.Org/cgi/reprint/20/4/252, accessed February 10, 2010). 

Murray CJL, Evans D, eds. Health systems performance assessment: debates, methods and empiricism. Chapter 
18: Monitoring the health financing function. Geneva, World Health Organization, 2003 (http://www.who.int/ 
health_financing/documents/cov-hspa, accessed February 10, 2010). 

Strategy on health care financing for countries of the Western Pacific and South-East Asia regions (2006-2010). 
New Delhi & Manila, World Health Organization, Regional Office for South-East Asia & Regional Office for 
the Western Pacific, 2005 (http://www.wpro.who.int/publications/PUB_929061210X.htm, accessed February 
10 , 2010 ). 

Resolution WHA58.33. Sustainable health financing, universal coverage and social health insurance. World 
Health Assembly. Geneva, World Health Organization, 2005 (http://www.who.int/health_financing/ 
documents/cov-wharesolution5833, accessed February 10, 2010). 

Tan-Torres Edejer T, et al., eds. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva, 
World Health Organization, 2003 (http://www.who.int/choice/book, accessed February 10, 2010). 

The impact of health expenditure on households and options for alternative financing. World Health Organization, 
Regional Committee for the Eastern Mediterranean, Fifty-first session, Cairo, 2004 (EM/RC51/4; http://www. 
who.int/health_financing/documents/cov-emrc-healthexpenditureimpact, accessed February 10, 2010). 

Resolution WHA56.25. The role of contractual arrangements in improving health systems’ performance. 
World Health Assembly. Geneva, World Health Organization, 2003 (http://gis.emro.who.int/ 
HealthSystemObservatory/PDF/Contracting/Introduction.pdf, accessed February 10, 2010). 

The world health report 2000 - health systems: improving performance. Geneva, World Health Organization, 
2000 (http://www.who.int/whr/2000, accessed February 10, 2010). 
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Financial catastrophe and impoverishment 

Xu K, et al. Designing health financing systems to reduce catastrophic health expenditure. Technical briefs for 
policy-makers. Geneva, World Health Organization, 2005 (http://www.who.int/health_financing/documents/ 
list/en/indexl .html, accessed February 10, 2010). 

Xu K, et al. Household catastrophic health expenditure: a multicountry analysis. The Lancet, 2003, 362:111-17 
(http://www.thelancet.com/journals/lancet/article/PIIS0140673603138615, accessed February 10, 2010). 

Kawabata K, Xu K, Carrin G. Preventing impoverishment through protection against catastrophic health 
expenditure. Bulletin of the World Health Organization, 2002, 80:612 (http://www.who.int/bulletin/archives/ 
volume80_8, accessed February 10, 2010). 

Related links 

GTZ-ILO-WHO-Consortium on Social Health Protection in Developing Countries (http://www. 
socialhealthprotection.org, accessed March 27, 2010). 

OECD data on health expenditures (http://www.oecd.org/document/16/0,3343 en_2649_34631_2085200_l_l 
_l_l,00.html, accessed March 27, 2010). 

WHO CHOosing Interventions that are Cost Effective (CHOICE) (http://www.who.int/choice, accessed March 
27, 2010). 

WHO Contractual arrangements in health systems (http://www.who.int/contracting, accessed March 27,2010). 

WHO’s Department of Health Systems Financing (http://www.who.int/healthsystems/financing, accessed 
March 27, 2010). 

WHO Health Financing Policy (http://www.who.int/health_financing/en/, accessed March 27, 2010). 

WHO Regional Office for Europe, health systems financing programme (http://www.euro.who.int/financing/, 
accessed March 27, 2010) 

WHO Regional Office for South-East Asia, Health systems development (http://searo.who.int/EN/Sectionl243/ 
Sectionl307.htm, accessed March 27, 2010). 

WHO Regional Office for the Western Pacific. Health financing and social protection (http://www.wpro.who.int/ 
sites/hcf/overview.htm, accessed March 27, 2010). 

WHO Simlns health financing policy tool (http://www.who.int/health_financing/tools/simins, accessed 
February 10,2010) 

World Bank: World Development Indicators (http://go.worldbank.org/AOLVGF2PWO, accessed March 29,2010). 
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6. Leadership and governance 



Leadership and governance 

6.1 Introduction 

Governance in health is being increasingly regarded as a salient theme on the development agenda. Leadership 
and governance in building a health system involve ensuring that strategic policy frameworks exist and are 
combined with effective oversight, coalition-building, regulation, attention to system design and accountability. 
The need for greater accountability arises both from increased funding and a growing demand to demonstrate 
results. Accountability is therefore an intrinsic aspect of governance that concerns the management of 
relationships between various stakeholders in health, including individuals, households, communities, firms, 
governments, nongovernmental organizations, private firms and other entities that have the responsibility to 
finance, monitor, deliver and use health services (1). Accountability involves, in particular: 

• delegation or an understanding (either implicit or explicit) of how services are supplied; 

• financing to ensure that adequate resources are available to deliver essential services; 

• performance around the actual supply of services; 

• receipt of relevant information to evaluate or monitor performance; 

• enforcement, such as imposition of sanctions or the provision of rewards for performance. 

Governance in health is a cross-cutting theme, intimately connected with issues surrounding accountability. 
In the context of health systems strengthening, it is an integral part of the health system components discussed 
in earlier sections of this handbook. Despite consensus on the importance of leadership and governance in 
improving health outcomes, they remain inadequately monitored and evaluated. 

6.2 Indicators for measuring health system governance 

Two types of indicators have been proposed for measuring governance: rules-based and outcome-based (2). 

Rules-based indicators measure whether countries have appropriate policies, strategies and codified approaches 
for health system governance. In the health systems context, these indicators include the existence, for 
example, of a national essential medicines list or a national policy on malaria control. They are part of a larger 
class of indicators called governance determinants (3). In addition to the existence of rules (called “formal 
procedures”), the determinants of health-care-provision governance include four other broad categories: 
ownership arrangements, decentralization, stakeholder participation, and contextual factors. In this framework, 
determinants of governance are contrasted with governance performance. 

Outcome-based indicators measure whether rules and procedures are being effectively implemented or enforced, 
based on the experience of relevant stakeholders. For health systems, examples may include the availability of 
essential medicines in health facilities or the absenteeism of health workers. Since the outcome-based indicators 
relate directly to the functioning of other health system “building blocks”, only the rules-based indicators for 
measuring health system governance are discussed in this section. 


@ Leadership and governance 


When selecting indicators for measuring governance in health, a high value should be placed on their usefulness 
and relevance. Nevertheless, even the most suitable governance indicators may be unable to adequately predict 
whether developments in a country or sector can be attributed to a change in governance. Thus, in general, 
governance indicators should not be used in isolation when designing policy responses to health system 
performance issues (4). 

6.3 Sources of information on health system governance 

Measurement of rules-based health system governance indicators will, in most cases, rely on both expert analysis 
of available sources such as administrative records (including legal/regulatory documents) coupled with expert 
reviews of national health policies. Administrative records are the important main data sources for rules- 
based indicators of governance and include legal and regulatory documents, national health strategies, budget 
documents, and regulations and guidelines that relate to the management, organization and financing of the 
health sector. Administrative records can be obtained from government publications, legal and administrative 
document departments and official web sites. 

The outcome-based governance indicators, which are discussed in other sections of this handbook, are generated 
using various data sources, including facility surveys, public expenditure reviews or client assessments. 

6.4 Core indicators 

A composite governance policy index, comprising 10 rules-based indicators that cover health policies for 
different disease interventions 1 and health system aspects, is presented. The index provides a summary 
measure of governance quality from a rules-based perspective. The indicators assess whether countries have 
policies, regulations and strategies in place to promote good leadership and governance in the health sector, but 
do not aim to assess enforcement. 

Each indicator is given a score of 0 if an adequate policy does not exist or cannot be assessed; and 1 if an 
adequate policy is available. The maximum score for the policy index is therefore 10. 

Each indicator is described below and summarized in Table 6.1. 

Recommended core indicator la: Existence of an up-to-date national health strategy 
linked to national needs and priorities 

Formulating national policies and strategies is a basic function of governments, and the task of formulating and 
implementing a health policy falls within the remit of the health ministry. An explicit health strategy defines 
the vision for the future, and outlines how objectives will be achieved. National health policies should outline 
priorities and the expected roles of different actors, inform and build consensus, and estimate the resources 
required to achieve goals and priorities. A recommended core indicator, therefore, is the existence of effective 
national health strategies and policies that reflect national needs and priorities, as opposed to factional political 
or financial interests, to foster broad-based political support and ownership of policies. 


1 Focusing particularly on diseases that are common in low-income and middle-income countries. 
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Recommended core indicator lb: Existence and year of last update of a published 
national medicines policy 

A NMP defines a framework for setting and monitoring medium- to long-term objectives in the public and 
private pharmaceutical sectors. It should encompass three objectives: (i) ensuring equitable availability and 
affordability of essential medicines; (ii) ensuring that all medicines are safe, efficacious and of high quality; 
and (iii) promoting rational use of medicines by health-care professionals and consumers. By attaining these 
objectives, countries can reduce morbidity and mortality, decrease the incidence of catastrophic illness that can 
increase impoverishment, and prevent large-scale losses to health and economic systems (5).Further elaboration 
of this indicator can be found in Section 4 of this handbook. 

Recommended core indicator lc: Existence of policies on medicines procurement that 
specify the most cost-effective medicines in the right quantities; open, competitive 
bidding of suppliers for quality products 

Expenditures on pharmaceuticals are highly susceptible to various forms of corruption. The pharmaceutical 
sector, with a global market value of over US$ 600 billion, is particularly vulnerable in the area of procurement. 
Procurement involves inventory management, aggregate purchasing, public bidding contests, technical analysis 
of offers, proper allocation of resources, payments, receipts of drugs purchased and quality control checks. These 
processes are often poorly documented and are thus a vulnerable target for corruption and fraud. Therefore to 
mitigate this threat, and to promote good governance, open bidding processes, good technical specifications 
and consistent and transparent procedures are essential. 

Recommended core indicator Id: Tuberculosis—existence of a national strategic plan 
for tuberculosis that reflects the six principal components of the Stop-TB strategy as 
outlined in the Global Plan to Stop TB 2006-2015 

Tuberculosis (TB) remains one of the world’s leading killers. In response to this global public health concern, 
WHO launched the Stop-TB strategy to assist countries in scaling up control activities, while also addressing 
the spread of TB-HIV co-infection and multidrug-resistant TB (MDR-TB).This indicator is therefore motivated 
by global TB control efforts to ensure that national TB plans are aligned with the six principal components of 
the Stop-TB strategy: 

Pursue high-quality DOTS expansion and enhancement (DOTS, or directly observed treatment, short-course, 
combines political commitment, microscopy services, drug supplies, surveillance and monitoring systems, 
and use of highly efficacious regimes with direct observation of treatment for TB). 

Address TB-HIV and MDR-TB and the needs of poor and vulnerable populations. 

Contribute to health system strengthening based on primary health care. 

Engage all care providers. 

Empower people with TB, and communities through partnership. 

Enable and promote research. 

Recommended core indicator le: Malaria—existence of a national malaria strategy or 
policy that includes drug efficacy monitoring, vector control and insecticide resistance 
monitoring 

WHO formulates global malaria policies and strategies. Vector control, i.e. the reduction of malaria morbidity 
and mortality through a reduction in the levels of mosquitoes, is generally the most effective intervention 
to prevent malaria transmission and therefore serves as one of the basic technical elements of the Global 
Malaria Control Strategy. Malaria control requires an integrated approach, involving prevention, treatment 
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with effective antimalarials, and monitoring and control at all levels. This indicator therefore monitors whether 
national malaria control programmes are aligned with the major priorities outlined by WHO, including drug 
efficacy monitoring, vector control and insecticide resistance monitoring. 

Recommended core indicator If: HIV/AIDS—completion of the UNGASS National 
Composite Policy Index questionnaire for HIV/AIDS 

At the close of the United Nations General Assembly Special Session (UNGASS) on HIV/AIDS in June 2001, 
189 Member States adopted the Declaration of Commitment on HIV/AIDS. This reflects the global consensus 
on a comprehensive framework to mitigate and control the spread of the HIV epidemic by 2015. An integral 
part of the core UNGASS indicators is the National Composite Policy Index, which reflects consensus among 
stakeholders on effective mechanisms for HIV/AIDS control. This index is designed to assess progress towards 
the development and implementation of national AIDS policies and strategies. It serves to track whether national 
policies and HIV/AIDS programmes are comprehensive and in line with the global priorities set forth by the 
UNGASS Declaration. In its annual country survey, the National Composite Policy Index includes questions 
on whether countries have developed a national multisectoral strategy or action framework to combat HIV/ 
AIDS and comprises, formal programme goals, clear targets and/or milestones, detailed budget of costs per 
programmatic area, indications of funding sources, and a monitoring and evaluation framework (6). 

Recommended core indicator Ig: Maternal health—existence of a comprehensive 
reproductive health policy consistent with the ICPD action plan 

The 1994 International Conference on Population and Development (ICPD) articulated a vision of the 
relationships between population, development and individual well-being. At the Conference, 179 governments 
adopted a 20-year plan of action, including reproductive health and rights, as well as women’s empowerment 
and gender equality as the cornerstone of population and development programmes. This indicator monitors 
whether reproductive health policies are both comprehensive and consistent with the ICPD plan of action. 

Recommended core indicator 7 h: Child health—existence of an updated 
comprehensive, multiyear plan for childhood immunization 

Immunization programmes are often based on past achievements and trends, with separate initiatives for 
each targeted disease, and too often seek to respond to specific global goals rather than to country needs and 
priorities. A comprehensive, multiyear plan for childhood immunization would face up to these challenges 
by proposing strategies that are all-inclusive and integrated with other health interventions. A comprehensive 
multiyear plan would evaluate the costs and financing options to ensure the financial sustainability of the 
programme and create linkages to broader health sector planning and budgeting processes. Such efforts would 
help to strengthen the capacity of countries to deliver immunization and child health services. 

Recommended core indicator 1i: Existence of key health sector documents that are 
disseminated regularly (such as budget documents, annual performance reviews and 
health indicators) 

The publication and dissemination of key health sector documents and reports, including annual budgets 
and performance reviews, promote accountability and transparency in the health sector. Such documentation 
helps to create an informed public and serves to improve government accountability to the population. A core 
indicator relating to the annual publication and dissemination of such materials seeks to create an environment 
that is responsive to public needs and concerns. 
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Recommended core indicator 7 j: Existence of mechanisms, such as surveys, for 
obtaining opportune client input on appropriate, timely and effective access to health 
services 

Surveys of patient satisfaction and utilization of health services are useful tools for obtaining information on the 
quality and responsiveness of health services. Such surveys may measure inputs (including whether facilities are 
properly equipped with essential medicines), processes (including whether waiting times are reasonable and 
treatment protocols are followed) and outcomes (including whether medical interventions reduce morbidity 
and mortality). Hence, an indicator that measures whether consumer satisfaction is taken into account in the 
assessment of health services reflects the responsiveness of health systems. 


Table 6.1 Summary of proposed indicators for health systems governance 


Indicators 

Data collection method 

Scoring 

Policy index 


Sum ofthe scores of 10 indicators. 

Max. score: 10 

la. Existence of an up-to-date national health strategy linked to 
national needs and priorities 

lb. Existence and year of last update ofa published national 
medicines policy 

lc. Existence of policies on medicines procurement that specify the 

Review of national health policies in 
respective domains (such as essential 
medicines and pharmaceutical, TB, 
malaria, HIV/AIDS, maternal health, 
child health/immunization). 

If adequate policy does not exist or 
cannot be assessed: 0 

If adequate policy is available: 1 


most cost-effective medicines in the right quantities; open, 
competitive bidding of suppliers of quality products 

l d. Tuberculosis—existence of a national strategic plan for 
tuberculosis that reflects the six principal components of the 
Stop-TB strategy as outlined in the Global Plan to StopTB 
2006-2015 

le. Malaria—existence ofa national malaria strategy or policy that 
includes drug efficacy monitoring, vector control and insecticide 
resistance monitoring 

l f. HIV/AIDS—completion ofthe UNGASS National Composite Policy 
Index questionnaire for HIV/AIDS 

lg. Maternal health—existence ofa comprehensive reproductive 
health policy consistent with the ICPD action plan 

1 h. Child health—existence of an updated comprehensive, multiyear 
plan for childhood immunization 

1 i. Existence of key health sector documents that are disseminated 
regularly (such as budget documents, annual performance 
reviews and health indicators) 

1 j. Existence of mechanisms, such as surveys, for obtaining 

opportune client input on appropriate, timely and effective access 
to health services 
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Selected tools and resources 


Citizen report card surveys: Washington DC, World Bank (http://siteresources.worldbank.org 
INTPCENG/1143380-1116506267488/20511066/reportcardnote.pdf, accessed 27 April 2010) 

These surveys are a mechanism to promote civil engagement and demand-side accountability, and 
empower individuals to express their views to government bodies. The surveys allow citizens to contribute 
to oversight and regulation and therefore aim to improve the quality and integrity of public services. 

Good governance for medicines: assessment instrument. Geneva, World Health Organization, 2006 (http://www. 
who.int/medicines/areas/policy/goodgovernance/GGM_assessment.pdf, accessed 11 April 2010). 

The WHO has initiated the Good Governance for Medicines (GGM) programme in an attempt to curb 
corruption in pharmaceutical sector systems by increasing transparency and accountability and promoting 
ethical practices. The GGM programme offers a three-step technical support package, namely national 
transparency assessment, development of national GGM programme, and implementation, to obtain a 
picture of the level of transparency and potential vulnerability to corruption in the public pharmaceutical 
sector using WHO’s assessment instrument. The assessment looks at six functions: registration of 
medicines, control of medicines promotion, inspection of establishments, selection of essential medicines, 
procurement and distribution. Licensing of establishments and control of clinical trials will be added soon. 

Measuring transparency in medicines registration, selection and procurement: four country assessment 
studies. Geneva, World Health Organization, AusAid, 2006 (http://www.who.int/medicines/areas/policy/ 
goodgovernance/WHO_PSM_PAR_2006.7.pdf, accessed 11 April 2010). 

Procurement (which involves inventory management, aggregate purchasing, public bidding contests, 
technical analysis of offers, proper allocation of resources, payments, receipts of drugs purchased and 
quality control checks), in particular, is vulnerable to corruption and fraud. In addition, the issue of 
counterfeit drugs has become salient as drugs are too often deliberately and fraudulently mislabelled with 
respect to their identity or source. Counterfeiting occurs both with branded and generic products, and 
counterfeit medicines may include products with the correct ingredients but fake packaging, with the 
wrong ingredients, without active ingredients, or with insufficient active ingredients. 

Further reading 

Anello E. Ethical infrastructure for good governance in the public pharmaceutical sector. Geneva, World Health 
Organization, 2006. 

Arndt C, Oman C. Uses and abuses of governance indicators. Paris, Organisation for Economic Co-operation and 
Development, 2006 (OECD Development Centre Studies) (http://www.oecd.org/dataoecd/21/16/40037762. 
pdf, accessed April 9, 2010). 

Bloom G, Standing H, Joshi A. Institutional arrangements and health service delivery in low-income countries. 
Brighton, Institute of Development Studies, 2006 (unpublished). 

Bloom G, Standing H, Joshi A. Institutional context of health services. In: Peters DH, Seharty E, Siadat B, 
Vujivic M, Janovsky K eds. Implementing health services strategies in low and middle income countries: from 
evidence to learning and doing. Washington DC, World Bank, 2009. 
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Bokhari F, Gai Y, Gottret P. Government health expenditures and health outcomes. Health Economics, 
2006,16:257-273. 

Chaudhury N, Hammer JS. Ghost doctors: absenteeism in Bangladeshi health facilities. Washington, DC, The 
World Bank, 2003 (World Bank Policy Research Working Paper No. 3065) (http://www-wds.worldbank.org/ 
external/default/WDSContentServer/WDSP/IB/2003/07/22/000094946_03070804210190/Rendered/PDF/ 
multiOpage.pdf, accessed April 9, 2010). 

Chopra M, et al. Effects of policy options on human resources for health: an analysis of systematic reviews. The 
Lancet, 2008, 371:668-674. 

Gottret P, Schieber G. Health financing revisited. Washington, DC, The World Bank, 2007. 

Immunization costing and financing: a tool and user guide for comprehensive multiyear planning. Geneva, World 
Health Organization/GAVI, 2006. 

International Health Regulations (IHR). Geneva, World Health Organization (http://who.int/topics/ 
international_health_regulations/en/, accessed 11 April 2010). 

International Health Regulations. Guidance for national policy-makers and partners. Geneva, World Health 
Organization, 2008 (http://www.who.int/csr/IHR%20Guidance%20for%20national%20policy%20makers%20 
and%20partners.pdf, accessed 11 May 2010). 

Kaufmann D, Kraay A, Zoido-Lobaton P. Aggregating governance indicators. Washington, DC, The World Bank, 
1999 (World Bank Research Working Paper No. 2195). 

Murray CJL, Evans DB. Health systems performance assessment. Geneva, World Health Organization, 2003. 

Saltman R, Ferrousier-David O. The concept of stewardship in health policy. Bulletin of the World Health 
Organization, 2000, 78:1-8. 

The Stop-TB strategy. Building on and enhancing DOTS to meet the TB-related Millennium Development Goals. 
Geneva, World Health Organization, 2006. 

The world health report 2000 - Health systems: improving performance. Geneva, World Health Organization, 
2000 . 

The world health report 2006 - Working together for health. Geneva, World Health Organization, 2006. 
Transparency International global corruption report 2006. London, Pluto Press, 2006. 

WHO medicines strategy - Countries at the core: 2004-2007, Geneva, World Health Organization, 2004. (http:// 
whqlibdoc.who.int/hq/2004/WHO_EDM_2004.5.pdf, accessed 11 April 2010). 
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